Signal processor

ABSTRACT

The present invention relates to a signal processing device which learns operations made by a user without the user knowing, and which can perform processing optimal to the user based on the learning results. At a learning unit  22,  operating signals supplied according to user operations are monitored, and judgment is made whether or not these can be used for learning. In the event that the operating signals are learning operating signals which can be used for learning, the learning unit  22  learns a correction norm which is the norm for correcting input signals, based on the learning operating signals. On the other hand, at a correcting unit  21,  post-correction signals, wherein the input signals are corrected based on the correction norm obtained by learning, are output as output signals. The present invention can be applied to an NR (Noise Reduction) circuit which removes noise.

TECHNICAL FIELD

The present invention relates to a signal processing device, andparticularly relates to a signal processing device which, for example,changes the contents of processing or the structure of processing byoperations made by the user, so as to perform optimal processing for theuser.

BACKGROUND ART

For example, with conventional NR (Noise Reduction) circuits, in theevent that a user operates an adjustment knob thereof and sets it at acertain position, noise removal processing corresponding to the positionof the knob is performed.

Now, the S/N (Signal to Noise Ratio) and frequency properties and thelike of signals input to the NR circuit are not necessarily alwaysconstant, rather, it is common for these to change. In the event thatthe S/N, frequency properties, etc., of the signals input to the NRcircuit change, suitable processing is not necessarily performed withregard to the signals input to the NR circuit in the noise removalprocessing corresponding to the position where the user has set theknob, and accordingly, the user needs to frequently operate the knob sothat suitable noise removal processing is performed for him/herself,which has been troublesome.

DISCLOSURE OF INVENTION

The present invention has been made in light of such, and accordingly,it is an object thereof to change the contents of processing or thestructure of processing by operations made by the user, so as to performoptimal processing for the user.

A first signal processing device according to the present inventioncomprises: signal processing means for performing signal processing oninput signals; and presenting means for presenting signal processingresults of the signal processing means, and internal information of thesignal processing means; wherein the structure of processing of thesignal processing means is changed based on operating signals suppliedaccording to user operations.

The signal processing means may comprise output signal generating meansfor generating output signals by linear combination of the input signalsand predetermined prediction coefficients; the presenting means maypresent the prediction coefficients as internal information of thesignal processing means; and the structure of processing of the signalprocessing means may be changed by-the prediction coefficients beingchanged based on the operating signals.

The signal processing means may further comprise: tutor data generatingmeans for generating tutor data from predetermined learning data;student data generating means for generating student data from thelearning data; and learning means for learning a prediction value of thetutor data obtained by linear combination of the student data and theprediction coefficients, and the prediction coefficients which minimizeerror with the tutor data; and the output signal generating means maygenerate the output signals using the prediction coefficients obtainedby learning with the learning means.

The learning means may learn the prediction coefficients byleast-N'th-power error which statistically minimizes an N'th-power-errorwhich is the error to the N'th power; and the structure of processing ofthe signal processing may be changed by the error to the N'th powerbeing changed based on the operating signals.

The signal processing means may further comprise storing means forstoring the prediction coefficients obtained by predetermined learningbeforehand.

The storing means may store multiple sets of the predictioncoefficients.

The prediction coefficients may be obtained by performing learning tominimize an error between a prediction value of tutor data serving as atutor for learning that is obtained by linear combination of studentdata serving as a student for learning and the prediction coefficients,and the tutor data; and the multiple sets of prediction coefficientsstored in the storing means may contain a set of prediction coefficientsobtained by learning by least-N'th-power error which statisticallyminimizes an N'th-power-error which is the error to the N'th power, anda set of prediction coefficients obtained by learning byleast-N'th-power error which statistically minimizes an N'th-power-errorwhich is different from N.

A first signal processing means according to the present inventioncomprises: a signal processing step for performing signal processing oninput signals; and a presenting step for presenting signal processingresults of processing in the signal processing step, and internalinformation of processing in the signal processing step; wherein thestructure of processing of the signal processing step is changed basedon operating signals supplied according to user operations.

A first program of a recording medium according to the present inventioncomprises: a signal processing controlling step for controlling signalprocessing on input signals; and a presentation controlling step forcontrolling presentation of signal processing results in the processingof the signal processing controlling step, and internal information inthe processing of the signal processing controlling step; wherein thestructure of processing of the signal processing controlling step ischanged based on operating signals supplied according to useroperations.

A first program according to the present invention causes a computer toexecute: a signal processing controlling step for controlling signalprocessing on input signals; and a presentation controlling step forcontrolling presentation of signal processing results in the processingof the signal processing controlling step, and internal information inthe processing of the signal processing controlling step; wherein thestructure of processing of the signal processing controlling step ischanged based on operating signals supplied according to useroperations.

A second signal processing device according to the present inventioncomprises: characteristics detecting means for detecting characteristicsfrom input signals; processing deciding means for deciding the contentsof processing with regard to the input signals, based on characteristicsdetected by the characteristics detecting means; processing executingmeans for executing processing with regard to the input signals,following the processing contents decided by the processing decidingmeans; and presenting means for presenting at least one of the internalinformation of the characteristics detecting means, the processingdeciding means, and the processing executing means; wherein theprocessing contents of at least one of the characteristics detectingmeans, the processing deciding means, and the processing executingmeans, changes, based on operating signals supplied according to useroperations.

The contents of processing of the characteristics detecting means may bechanged based on the operating signals.

The operating signals may be signals specifying, of multiple types ofcharacteristics, a predetermined number of types of characteristics; andthe characteristics detecting means may change the contents ofprocessing thereof so as to detect the predetermined number of types ofcharacteristics.

The characteristics detecting means may detect the predetermined numberof types of characteristics from the input signals; and the processingdeciding means may decide the contents of processing at the processingexecuting means with regard to the input signals, based on thepredetermined number of types of characteristics detected from the inputsignals by the characteristics detecting means.

The operating signals may be signals specifying which of a first orsecond processing object the input signals are; and the presenting meansmay present a distribution of characteristics of the input signalsserving as the first and second processing objects, as the internalinformation.

In the event that there is input of the operating signals specifyingwhich of a first or second processing object the input signals are, withregard to the distribution presented by the presenting means, thecontents of processing performed on the input signals may be changed,based on the operating signals.

The input signals may be image signals; the processing deciding meansmay decide whether or not to output the input signals without change, asthe processing contents with regard to the input signals, based on thepredetermined number of types of characteristics detected from the inputsignals by the characteristics detecting means; and the processingexecuting means may detect a telop within image signals which are theinput signals, by selectively outputting the input signals following thedecision made by the processing deciding means.

The processing contents of the processing deciding means may be changedbased on the operating signals.

The processing deciding means may comprise storing means for storing atable indicating the correlation relationship between thecharacteristics and the processing contents with regard to the inputsignals having the characteristics; the processing deciding means maydecide the processing contents correlated on the table to thecharacteristics of the input signals detected by the characteristicsdetecting means, as processing contents for the input signals; and thepresenting means may present the table as internal information of theprocessing deciding means.

The processing deciding means may change the processing contents ofitself, by changing the processing contents in the table, based on theoperating signals.

Processing for outputting output signals of a first value, andprocessing for outputting output signals of a second value, may exist inthe table as processing contents with regard to the input signals; andthe processing executing means may binarize the input signals into thefirst and second values, following the decision of the processingdeciding means.

The processing contents of the processing executing means may be changedbased on the operating signals.

The processing executing means may generate output signals by linearcombination of the input signals and predetermined predictioncoefficients; and may change the contents of processing thereof, bychanging the prediction coefficients based on the operating signals.

The prediction coefficients linearly combined with the input signals maybe changed so that the sum thereof is 1.

Of prediction coefficients other than prediction coefficients changedbased on the operating signals of the array of prediction coefficientslinearly combined with the input signals, those close to predictioncoefficients changed based on the operating signals may be changed inthe same direction as the prediction coefficients changed based on theoperating signals, and those distant from prediction coefficientschanged based on the operating signals may be changed in the oppositedirection as the prediction coefficients changed based on the operatingsignals.

Of prediction coefficients other than prediction coefficients changedbased on the operating signals of the array of prediction coefficientslinearly combined with the input signals, those having extreme valuewith the same polarity as prediction coefficients changed based on theoperating signals may be changed in the same direction as the predictioncoefficients changed based on the operating signals, and those havingextreme value with a different polarity as prediction coefficientschanged based on the operating signals changed in the opposite directionas the prediction coefficients may be changed based on the operatingsignals.

A second signal processing-method according to the present inventioncomprises: a characteristics detecting step for detectingcharacteristics from input signals; a processing deciding stepfor-deciding the contents of processing with regard to the inputsignals, based on characteristics detected in the characteristicsdetecting step; a processing executing step for executing processingwith regard to the input signals, following the processing contentsdecided in the processing deciding step; and a presenting step forpresenting at least one of the internal information of processing in thecharacteristics detecting step, processing in the processing decidingstep, and processing in the processing executing step; wherein theprocessing contents of at least one of the processing in characteristicsdetecting step, processing in the processing detecting step, andprocessing in the processing executing step, changes, based on operatingsignals supplied according to user operations.

A program of a second recording medium according to the presentinvention comprises: a characteristics detection controlling step forcontrolling detection of characteristics from input signals; aprocessing decision controlling step for controlling decision of thecontents of processing with regard to the input signals, based oncharacteristics detected by the processing in the characteristicsdetection controlling step; a processing execution controlling step forcontrolling execution of processing with regard to the input signals,following the processing contents decided by the processing in theprocessing decision controlling step; and a presentation controllingstep for controlling presentation of at least one of the internalinformation of processing in the characteristics detection controllingstep, processing in the processing decision controlling step, andprocessing in the processing execution controlling step; wherein theprocessing contents of at least one of the processing in characteristicsdetection controlling step, processing in the processing decisioncontrolling step, and processing in the processing execution controllingstep, changes, based on operating signals supplied according to useroperations.

A second program according to the present invention causes a computer toexecute: a characteristics detection controlling step for controllingdetection of characteristics from input signals; a processing decisioncontrolling step for controlling decision of the contents of processingwith regard to the input signals, based on characteristics detected bythe processing in the characteristics detection controlling step; aprocessing execution controlling step for controlling execution ofprocessing with regard to the input signals, following the processingcontents decided by the processing in the processing decisioncontrolling step; and a presentation controlling step for controllingpresentation of at least one of the internal information of processingin the characteristics detection controlling step, processing in theprocessing decision controlling step, and processing in the processingexecution controlling step; wherein the processing contents of at leastone of the processing in characteristics detection controlling step,processing in the processing decision controlling step, and processingin the processing execution controlling step, changes, based onoperating signals supplied according to user operations.

A third signal processing device according to the present inventioncomprises: output value generating means for generating output values bylinear combination of input signals and a plurality of coefficientsstored beforehand; and presenting means for presenting the output valuesand the plurality of coefficients; wherein one or more of the pluralityof coefficients is changed based on operating signals supplied accordingto user operations.

The coefficients linearly combined with the input signals are may bechanged so that the sum thereof is 1.

In the event that one of the plurality of coefficients is changed basedon the operating signals, of coefficients other than coefficientschanged based on the operating signals of the array of the plurality ofcoefficients, those close to coefficients changed based on the operatingsignals may be changed in the same direction as the coefficients changedbased on the operating signals, and those distant from coefficientschanged based on the operating signals may be changed in the oppositedirection as the coefficients changed based on the operating signals.

In the event that one of the plurality of coefficients is changed basedon the operating signals, of coefficients other than coefficientschanged based on the operating signals of the array of the plurality ofcoefficients, those having extreme value with the same polarity ascoefficients changed based on the operating signals may be changed inthe same direction as the coefficients changed based on the operatingsignals, and those having extreme value with a different polarity ascoefficients changed based on the operating signals may be changed inthe opposite direction as the coefficients changed based on theoperating signals.

The plurality of coefficients may be obtained by performing learningwherein an error between a prediction value of tutor data serving as atutor for learning which is obtained by linear combination of studentdata serving as a student for learning and the plurality ofcoefficients, and the tutor data thereof, is minimized.

A third signal processing method according to the present inventioncomprises: an output value generating step for generating output valuesby linear combination of input signals and a plurality of coefficientsstored beforehand; and a presenting step for presenting the outputvalues and the plurality of coefficients; wherein one or more of theplurality of coefficients is changed based on operating signals suppliedaccording to user operations.

A program of a third recording medium according to the present inventioncomprises: an output value generation controlling step for controllinggeneration of output values by linear combination of input signals and aplurality of coefficients stored beforehand; and a presentationcontrolling step for controlling presentation of the output values andthe plurality of coefficients; wherein one or more of the plurality ofcoefficients is changed based on operating signals supplied according touser operations.

A third program according to the present invention causes a computer toexecute: an output value generation controlling step for controllinggeneration of output values by linear combination of input signals and aplurality of coefficients stored beforehand; and a presentationcontrolling step for controlling presentation of the output values andthe plurality of coefficients; wherein one or more of the plurality ofcoefficients is changed based on operating signals supplied according touser operations.

With the first invention, input signals are subjected to signalprocessing, the results of signal processing and internal informationare presented, and the structure of processing of the signals processingmeans is changed based on operating signals supplied according tooperations made by the user.

With the second invention, characteristics are detected from inputsignals, the contents of processing with regard to the input signals aredecided based on the detected characteristics, processing on the inputsignals is executed following the contents of processing decided upon,the internal information of at least one of characteristics detecting,processing deciding, or processing execution, is presented, and thecontents of processing of at least one of characteristics detecting,processing deciding, or processing execution, is changed based onoperating signals supplied according to operations made by the user.

With the third invention, output values are generated by linearcombination of input signals and multiple coefficients storedbeforehand, the output values and multiple coefficients are presented,and one or more of the multiple coefficients are changed based onoperating signals supplied according to operations made by the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an optimizing device to which thepresent invention has been applied.

FIG. 2 is a block diagram illustrating a configuration example of anembodiment of an optimizing device to which the present invention hasbeen applied.

FIG. 3 is a flowchart describing optimizing processing with theoptimizing device shown in FIG. 2.

FIG. 4 is a block diagram illustrating a configuration example of anembodiment of an NR circuit using an optimizing device.

FIG. 5A is a waveform diagram representing input signals.

FIG. 5B is a waveform diagram representing input reliability.

FIG. 6 is a flowchart describing correction processing with an NRcircuit.

FIG. 7 is a flowchart describing correction parameter computingprocessing with an NR circuit.

FIG. 8 is a flowchart describing control data learning processing withan NR circuit.

FIG. 9A through FIG. 9C are diagrams describing control data learningprocessing.

FIG. 10 is a block diagram illustrating a configuration example ofanother embodiment of an NR circuit using an optimizing device.

FIG. 11 is a diagram illustrating pixels multiplied with parametercontrol data.

FIG. 12 is a flowchart describing correction parameter computingprocessing with an NR circuit.

FIG. 13 is a flowchart describing control data learning processing withan NR circuit.

FIG. 14 is a block diagram illustrating a configuration example ofanother embodiment of an NR circuit using an optimizing device.

FIG. 15 is a flowchart describing optimizing processing with theoptimizing device shown in FIG. 14.

FIG. 16 is a block diagram illustrating a configuration example of anembodiment of an auto-driving device to which the present invention hasbeen applied.

FIG. 17 is a block diagram illustrating a configuration example of theprocessing unit of the optimizing device shown in FIG. 16.

FIG. 18 is a flowchart describing correction parameter computingprocessing with the optimizing device shown in FIG. 16.

FIG. 19 is a flowchart describing control data learning processing withthe optimizing device shown in FIG. 16.

FIG. 20 is a block diagram illustrating another configuration example ofthe processing unit of the optimizing device shown in FIG. 16.

FIG. 21 is a diagram illustrating a driving direction output by thecomputing unit shown in FIG. 16

FIG. 22 is a flowchart describing the correction processing by theoptimizing device shown in FIG. 16.

FIG. 23 is a flowchart describing the correction parameter learningprocessing by the optimizing device shown in FIG. 16.

FIG. 24 is a block diagram illustrating another configuration example ofan auto-driving device to which the present invention has been applied.

FIG. 25 is a flowchart describing correction parameter computingprocessing by the optimizing device shown in FIG. 24.

FIG. 26G is a flowchart describing correction parameter learningprocessing by the optimizing device shown in FIG. 24.

FIG. 27 is a diagram illustrating an example of internal informationgenerated by the internal information generating unit shown in FIG. 24.

FIG. 28 is a diagram illustrating an example of internal informationgenerated by the internal information generating unit shown in FIG. 24.

FIG. 29 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 30 is a block diagram illustrating a configuration example of thelearning unit of the optimizing device shown in FIG. 29.

FIG. 31 is a block diagram illustrating a configuration example of themapping processing unit of the optimizing device shown in FIG. 29.

FIG. 32 is a diagram describing error between a true value and aprediction value.

FIG. 33 is a diagram describing the least-N'th-power error method.

FIG. 34 is a diagram describing weight α_(s).

FIG. 35 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 29.

FIG. 36 is a diagram illustrating a comparison between aleast-N'th-power norm and a least-square norm.

FIG. 37 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 38 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 37.

FIG. 39 is a diagram illustrating an example of internal informationgenerated by the internal information generating unit shown in FIG. 37.

FIG. 40 is a diagram illustrating an example of internal informationgenerated by the internal information generating unit shown in FIG. 37.

FIG. 41 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 42 is a block diagram illustrating a configuration example of thecoefficient converting unit of the optimizing device shown in FIG. 41.

FIG. 43 is a block diagram illustrating a configuration example of alearning device which generates coefficients stored in the coefficientmemory shown in FIG. 41 by learning.

FIG. 44 is a flowchart describing the coefficient deciding processing bythe learning device shown in FIG. 43.

FIG. 45 is a diagram illustrating the configuration of a prediction tap.

FIG. 46 is a diagram illustrating an example of the distribution ofcoefficient values corresponding to the tap locations of the predictiontap.

FIG. 47 is a diagram illustrating the configuration of a prediction tap.

FIG. 48 is a diagram illustrating an example of the distribution ofcoefficient values corresponding to the tap locations of the predictiontap.

FIG. 49 is a diagram illustrating the configuration of a prediction tap.

FIG. 50 is a diagram illustrating a spring model.

FIG. 51 is a diagram illustrating a equilibrium model.

FIG. 52 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 41.

FIG. 53 is a block diagram illustrating a configuration of anotherembodiment of an optimizing device to which the present invention hasbeen applied.

FIG. 54 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 53.

FIG. 55 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 56 is a block diagram illustrating a configuration example of thefeatures detecting unit shown in FIG. 55.

FIG. 57 is a block diagram illustrating a configuration example of theprocessing deciding unit shown in FIG. 55.

FIG. 58 is a block diagram illustrating a configuration example of theprocessing unit shown in FIG. 55.

FIG. 59 is a flowchart describing telop extraction optimizing processingby the optimizing device shown in FIG. 55.

FIG. 60 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 61 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 60.

FIG. 62 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 63 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 64 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 65 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 66 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 67 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 68 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 62.

FIG. 69 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 70 is a block diagram illustrating a configuration example of theprocessing deciding unit shown in FIG. 69.

FIG. 71 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 72 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 73 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 74 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 75 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 76 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 77 is a diagram describing telop extraction optimizing-processingwith the optimizing device shown in FIG. 69.

FIG. 78 is a diagram describing telop extraction optimizing processingwith the optimizing device shown in FIG. 69.

FIG. 79 a diagram describing switching over of features with theoptimizing device shown in FIG. 69.

FIG. 80 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 81 is a block diagram illustrating a configuration example of thefeatures detecting unit shown in FIG. 80.

FIG. 82 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 80.

FIG. 83 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 84 is a block diagram illustrating a configuration example of thefeatures detecting unit shown in FIG. 80.

FIG. 85 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 84.

FIG. 86 is a diagram describing an alteration contents instructingscreen for features, with the optimizing device shown in FIG. 84.

FIG. 87 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has-been applied.

FIG. 88 is a flowchart describing telop extraction optimizing processingwith the optimizing device shown in FIG. 87.

FIG. 89 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 90 is a block diagram illustrating a configuration example of theprocessing deciding unit shown in FIG. 89.

FIG. 91 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 84.

FIG. 92 is a diagram illustrating an LUT.

FIG. 93 is a diagram describing the processing contents specified foreach of the features on the LUT.

FIG. 94 is a diagram describing the processing contents specified foreach of the features on the LUT.

FIG. 25 is a diagram describing the processing contents specified foreach of the features on the LUT.

FIG. 96 is a flowchart describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 97 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 98 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 99 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 100 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 101 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 102A is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 102B is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 103A is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 103B is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 104A is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 104B is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

FIG. 105 is a flowchart describing auto LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 106 is a diagram describing auto LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 107 is a diagram describing auto LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 108 is a diagram describing auto LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 109 is a diagram describing auto LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 91.

FIG. 110 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 111 is a block diagram illustrating a configuration example of theprocessing deciding unit shown in FIG. 110.

FIG. 112 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 110.

FIG. 113 is a flowchart describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG.110.

FIG. 114 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 110.

FIG. 115 is a diagram describing manual LUT changing processing in imageoptimizing processing with the optimizing device shown in FIG. 110.

FIG. 116 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 117 is a block diagram illustrating a configuration example of theprocessing unit shown in FIG. 116.

FIG. 118 is a block diagram of a learning device for generatingcoefficient sets stored in the coefficient memory shown in FIG. 116 bylearning.

FIG. 119 is a block diagram illustrating a configuration example of themapping processing unit shown in FIG. 117.

FIG. 120 is a flowchart describing learning processing with theoptimizing device shown in FIG. 116.

FIG. 121 is a flowchart describing mapping processing with theoptimizing device shown in FIG. 116.

FIG. 122 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 123 is a block diagram illustrating a configuration example of theprocessing unit shown in FIG. 122.

FIG. 124 is a flowchart describing learning processing by the optimizingdevice shown in FIG. 122.

FIG. 125 is a flowchart describing mapping processing by the optimizingdevice shown in FIG. 122

FIG. 126 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 127 is a block diagram illustrating a configuration example of theprocessing unit shown in FIG. 126.

FIG. 128 is a block diagram of a learning device for generatingcoefficient sets stored in the coefficient memory shown in FIG. 127 bylearning.

FIG. 129 is a flowchart describing coefficients deciding processing withthe learning device shown in FIG. 128.

FIG. 130 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 126.

FIG. 131 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

FIG. 132 is a block diagram illustrating a configuration example of theprocessing unit shown in FIG. 131.

FIG. 133 is a flowchart describing image optimizing processing with theoptimizing device shown in FIG. 131.

FIG. 134 is a block diagram illustrating a configuration example of anembodiment of a computer to which the present invention has beenapplied.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 illustrates a configuration example of an embodiment of anoptimizing device to which the present invention has been applied.Following executing predetermined processing with regard to inputsignals (signal processing), the optimizing device outputs signalsobtained as the results of the processing, as output signals. The userexamines (qualitatively evaluates) the output signals, and in the eventthat the output signals do not match personal preferences, inputsoperating signals corresponding to the preferences of the user, to theoptimizing device. The optimizing device changes the contents ofprocessing and structure of processing based on the operating signals,subjects the input signals to the predetermined processing again, andoutputs the output signals, The optimizing device subjects the inputsignals to optimal processing (processing which is optimal for the user)while repeating changing the contents of the processing and structure ofprocessing corresponding to operating signals input by user operations,thereby outputting output signals which are closer to the preferences ofthe user.

FIG. 2 illustrates a first detailed configuration example of theoptimizing device shown in FIG. 1.

With this optimizing device 1, processing which is optimal for the useris performed by learning the operations of the user without the userknowing. That is, with the optimizing device, operating signals suppliedcorresponding to operations of the user are monitored, and judgment ispassed regarding whether or not these can be used for learning. In theevent that the operating signals are learning operating signals whichcan be used for learning, a correction norm which is the norm forcorrecting the input signals is learned, based on the learning operatingsignals. On the other hand, input signals are corrected based on thecorrection norm obtained by learning, and signals following thecorrection are output as output signals.

The optimizing device 1 is configured of a processing unit 11 made up ofa correcting unit 21 and a learning unit 22, to which are suppliedoperating signals corresponding to user operations, besides inputsignals which are subject to processing.

The operating signals are supplied from an operating unit 2. That is,the operating unit 2 is configured of, for example, rotary or slideknobs, switches, pointing devices, etc., and operating signalscorresponding to user operations are supplied to the processing unit 11making up the optimizing device 1.

Digital input signals, for example, are supplied to the correcting unit21 making up the optimizing device 1, as well as, for example,correction parameters serving as correction norm for correcting inputsignals, from the learning unit 22. The correcting unit 21 correctsinput signals based on the correction parameters (signal processing),and outputs signals following the correction, as output signals.

As well as operating signals from the operating unit 2 being supplied tothe learning unit 22, input signals or output signals are also suppliedthereto, as necessary. The learning unit 22 monitors the operatingsignals, and judges whether or not these can be used for learning. Inthe event that the operating signals are learning operating signalswhich can be used for learning, the learning unit 22 learns correctionparameters used for correcting input signals using the input signals orthe output signals based on the learning operating signals, asnecessary, and supplies these to the correcting unit 21.

Note that the learning unit 22 has built-in learning data memory 53 andlearning information memory 55, wherein the learning data memory 53stores learning data used for learning, and the learning informationmemory 55 stores later-described learning information obtained bylearning.

Next, the processing (optimizing processing) which the optimizing device1 shown in FIG. 2 performs, will be described with reference to theflowchart shown in FIG. 3.

First, in step S1, the learning unit 22 judges whether or not learningoperating signals have been received from operating unit 2. Here, in theevent of operating the operating unit 2, usually, the user firstperforms general operations, and then performs detailed operations whileconfirming the output signals output according to the operations, andthen stops operations at the point that output signals which the userthinks are ultimately optimal are obtained. The operating signalscorresponding to the position of the operating unit 2 at the point whereoutput signals which the user thinks are optimal are obtained are thelearning operating signals, and accordingly, in the event that theoperating unit 2 is operated for a predetermined amount of time orlonger and then the operating is stopped, the learning unit 22 judgesthe operating signals at the time of stopping as being learningoperating signals.

In step S1, in the event that judgement is made that learning operatingsignals are not received, i.e., in the event that the user has notoperated the operating unit 2, or has operated but is still looking foran optimal position, for example, step S2 through step S10 are skipped,the flow proceeds to step S11, and the correcting unit 21 corrects theinput signals according to the correction parameters already set, outputsignals are output as the correction results, and the flow returns tostep S1.

Also, in step S1, in the event that judgement is made that learningoperating signals have been received, the flow proceeds to step S2, thelearning unit 22 obtains learning data used for learning, based on thelearning operating signals, and the flow proceeds to step S3. In stepS3, the learning data memory 53 stores the newest learning data obtainedin step S2.

Here, the learning data memory 53 has storage capacity capable ofstoring multiple sets of learning data. Also, in the event that thisstorage capacity is filled up, the learning data memory 53 is arrangedso as to overwrite the oldest stored values with the next learning data.Accordingly, there are multiple sets of learning data in the learningdata memory 53, with the newest stored.

Following storing of the learning data in the learning data memory 53 instep S3, the flow proceeds to step S4, where the learning unit 22performs learning using the newest learning data stored in the learningdata memory 53 and the learning information stored in the learninginformation memory 55, obtains correction parameters, and proceeds tostep S5. In step S5, the learning unit 22 updates the stored contents ofthe learning information memory 55 with new learning informationobtained part way through the learning in step S4, and the flow proceedsto step S6.

In step S6, the learning unit 22 obtains a later-described propernesswhich represents the properness of the correction parameters obtained instep S4, the flow proceeds to step S7, and judgment is made regardingwhether or not the correction parameters obtained in step S4 are proper,based on the properness.

In the event that judgement is made in step S7 that the correctionparameters are proper, steps S8 and S9 are skipped, the flow proceeds tostep S10, and a learning unit 22 outputs the correction parametersjudged to be proper to the correcting unit 21, and the flow proceeds tostep S11. Accordingly, in this case, subsequent input signal correctionis performed at the correcting unit 21 following the correctionparameters newly obtained by the learning in step S4.

On the other hand, in the event that judgment is made in step S7 thatthe correction parameters are not proper, the flow proceeds to step S8,the learning unit 22 performs learning using only the newest learningdata of the learning data stored in the learning data memory 53, obtainscorrection parameters, and proceeds to step S9. In step S9, the learningunit 22 updates the stored contents of the learning information memory55 with the new learning information obtained part-way through thelearning in step S8, and the flow proceeds to step S10. In this case, instep S10, the learning unit 22 outputs the correction parametersobtained only from the newest learning data in step S8 to the correctingunit 21, and the flow proceeds to step S11. Accordingly, in this case,subsequent input signal correction is performed at the correcting unit21 following the correction parameters newly obtained by the learning instep S8.

Next, FIG. 4 illustrates a detailed configuration example of a casewherein the processing unit 11 shown in FIG. 2 is applied to an NRcircuit which removes noise from image signals and audio signals, forexample.

Weight memory 31 stores a weight (coefficient) W (e.g., a value that is0 or greater but 1 or smaller) as correction parameters supplied from alater-described selecting unit 41 of the learning unit 22. The weightmemory 32 stores weight 1−W supplied from the computing unit 33.

The computing unit 33 supplies a subtracted value 1−W wherein the weightw supplied from the selecting unit 41 of the learning unit 22 issubtracted from 1.0, which is supplied to the weighting memory 32 asweight. The computing unit 34 multiplies the input signals and theweight 1−W stored in the weight memory 32, and supplies the multipliedvalue to a computing unit 36. A computing unit 35 multiplies the weightW stored in the weight memory 31 and the output signals stored (latched)at the latch circuit 37, and supplies the multiplied value thereof tothe computing unit 36. The computing unit 36 adds the outputs of thecomputing units 34 and 35, and outputs the added value thereof as outputsignals.

A latch circuit 37 latches on the output signals which the computingunit 36 outputs, and supplies these to the computing unit 35.

With the embodiment shown in FIG. 4, the correcting unit 21 of theprocessing unit 11 is configured of the above weight memory 31 and 32,computing units 33, 34, 35, and 36, and the latch circuit 37.

This selecting unit 41 selects one or the other of the weight output bythe weight correcting unit 46 or the weight output by the operatingsignal processing unit 50, and supplies this to the correcting unit 21as correction parameters.

The input reliability calculating unit 42 is arranged so as to besupplied with input signals, and obtains the input reliability whichrepresents the reliability of the input signals thereof, which areoutput to the output reliability calculating unit 43 and the weightingcalculating unit 45. The output reliability calculating unit 43 obtainsoutput reliability representing the reliability of the output signals,based on the input reliability from the input reliability calculatingunit 42, and supplies this to the latch circuit 44 and the weightingcalculating unit 45. The latch circuit 44 stores (latches) the outputreliability from the output reliability computing unit 43, and suppliesthis to the output reliability calculating unit 43 and the weightingcalculating unit 45.

The weighting calculating unit 45 calculates the weighting from theinput reliability from the input reliability calculating unit 42 and theoutput reliability from the output reliability calculating unit 43, andoutputs this to the weighting correcting unit 46. The weightingcorrecting unit 46 is arranged so as to be supplied from parametercontrol data memory 57 with parameter control data which controls theweighting serving as correction parameters, in addition to weight, andthe weighting correcting unit 46 processes (corrects) the weighting withthe parameter control data, and supplies this to the selecting unit 41.

An operating signal processing unit 50 is arranged so as to be suppliedwith operating signals from the operating unit 2 (FIG. 2), with theoperating signal processing unit 50 processing the operating signalssupplied thereto, and supplying the weight corresponding to theoperating signals thereof to the selecting unit 41, a tutor datagenerating unit 51, and a student data generating unit 52. Further, theoperating signal processing unit 50 judges whether or not the operatingsignals are the above-described learning operating signals, and in theevent that the operating signals are learning operating signals, a flagto that effect (hereafter may be referred to as learning flag) is addedto the weight to be output.

Upon receiving the weight with a learning flag added thereto from theoperating signal processing unit 50, the tutor data generating unit 51generates tutor data which serves as a tutor for learning, and suppliesthis to the learning data memory 53. That is, the tutor data generatingunit 51 supplies the weight to which a learning flag has been added, tothe learning data memory 53, as tutor data.

The student data generating unit 52, upon receiving the weight with thelearning flag added from the operating signal processing unit 50,generates students data to serve as a student for learning, and suppliesthis to the learning data memory 53. That is, the student datagenerating unit 52 is configured in the same way as the above-describedinput reliability calculating unit 42, output reliability calculatingunit 43, latch circuit 44, and weighting calculating unit 45, tocalculate weighting from input signals supplied thereto, and supplyingthe weighting calculated from the input signals upon reception ofweighting with a learning flag added thereto to the learning data memory53, as student data.

The learning data memory 53 stores a set of tutor data serving asweighting corresponding to learning operating signals, supplied from thetutor data generating unit 51, and student data serving as weightingcalculated from the input signals at the time of reception of thelearning operating signals, supplied from the student data generatingunit 52, as one set of learning data. As described above, the learningdata memory 53 is capable of storing multiple sets of learning data, andfurther, upon storing learning data to the storage capacity, is arrangedso as to store the next learning data by overwriting the oldest storedvalues. Accordingly, the learning data memory 53 basically has severalof the newest learning data sets stored therein at all times.

A parameter control data calculating unit 54 learns the tutor data andstudent data serving as learning data stored in the learning data memory53, as well as, if necessary, parameter control data which minimizes apredetermined statistical error by computing new learning informationusing the learning information stored in the learning information memory55, under control of a judgment control unit 56, which is supplied tothe judgment control unit 56. Also, the parameter control datacalculating unit 54 updates the stored contents of the learninginformation memory 55 with new learning information obtained bylearning. The learning information memory 55 stores the learninginformation from the parameter control data calculating unit 54.

The judgment control unit 56 judges the properness of the parametercontrol data supplied from the parameter control data calculating unit54, by making reference to the newest learning data stored in thelearning data memory 53. Also, the judgment control unit 56 controls theparameter control data calculating unit 54 and supplies the parametercontrol data supplied from the parameter control data calculating unit54, to the parameter control data memory 57. The parameter control datamemory 57 updates the stored contents thereof by the parameter controldata supplied from the judgment control unit 56, and supplies this tothe weighting correcting unit 46.

With the embodiment shown in FIG. 4, the learning unit 22 of theprocessing unit 11 is configured of the above-described selecting unit41 through weighting correcting unit 46, and the operating signalprocessing unit 50 through parameter control data memory 57.

With the processing unit 11 of the optimizing device 1 serving as an NRcircuit configured as described above, noise in input signals is removedas follows.

That is, for example, to simplify description, considering removingnoise which fluctuates over time with regard to input signals having aconstant true value upon which noise which fluctuates over time issuperimposed, as shown in FIG. 5A, by taking the average thereof, noisecan be effectively removed by making the weighting small (not givingmuch consideration) for input signals with a high noise level (i.e.,signals with poor S/N), for example, as the degree of noise, whilemaking the weighting great for input signals with a low noise level(i.e., signals with good S/N).

Accordingly, with the NR circuit shown in FIG. 4, noise can beeffectively removed by obtaining the closeness of the input signals tothe true value as an evaluation value of input signals, as shown in FIG.5B for example, i.e., the input reliability representing the reliabilitythat the input signals are the true value, and calculating the averagethereof by performing weighting corresponding to the input reliabilityfor the input signals.

Accordingly, with the NR circuit shown in FIG. 4, the weight averageusing the weighting corresponding to the input reliability thereof isobtained with regard to the input signals, and output as output signals,and the output signals y(t) can be obtained with the followingExpression, wherein the input signals, the output signals, and inputreliability, at time t, are represented by x(t), y(t), and α_(x(t)).$\begin{matrix}{\lbrack {{Expression}\quad 1} \rbrack\quad{{y(t)} = \frac{\sum\limits_{i = 0}^{t}{\alpha_{x{(i)}}{x(i)}}}{\sum\limits_{i = 0}^{t}\alpha_{x{(i)}}}}} & (1)\end{matrix}$

Here, the greater the input reliability α_(x(t)) is, the greater theweighting provided is.

From Expression (1), the output signal y(t-1) from one sample back fromthe current time t can be obtained from the following Expression.$\begin{matrix}{\lbrack {{Expression}\quad 2} \rbrack\quad{{y( {t - 1} )} = \frac{\sum\limits_{i = 0}^{t - 1}{\alpha_{x{(i)}}{x(i)}}}{\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}}}} & (2)\end{matrix}$

Also, with regard to the output signals y(t) as well, output reliabilityα_(y(t)) which represents the closeness to the true value, i.e., thereliability that the output signals y(t) are the true value, isintroduced as the evaluation value of the output signals y(t), and theoutput reliability α_(y(t-1)) for the output signal y(t-1) from onesample back from the current time t is defined by the followingExpression. $\begin{matrix}{\lbrack {{Expression}\quad 3} \rbrack\quad{{\alpha_{y}( {t - 1} )} = {\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}}}} & (3)\end{matrix}$

In this case, from the Expressions (1) through (3), the output signalsy(t) and the output reliability α_(y(t)) thereof can be expressed asfollows. $\begin{matrix}{\lbrack {{Expression}\quad 4} \rbrack\quad{{y(t)} = {\frac{\overset{\overset{\alpha_{y{({t - 1})}}{y{({t - 1})}}}{︷}}{\sum\limits_{i = 0}^{t - 1}{\alpha_{x{(i)}}{x(i)}}} + {\alpha_{x{(t)}}{x(t)}}}{\underset{\underset{\alpha_{y{({t - 1})}}}{︸}}{\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}} + \alpha_{x{(t)}}}\quad = \frac{{\alpha_{y{({t - 1})}}{y( {t - 1} )}} + {\alpha_{x{(t)}}{x(t)}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}}} & (4)\end{matrix}$[Expression 5]α_(y(t))=α_(y(t-1))+α_(x(t))   (5)

Also, the weight used for obtaining the output signal y(t) at the time tis represented by w(t), and is defined by the following Expression.$\begin{matrix}{\lbrack {{Expression}\quad 6} \rbrack\quad{{w(t)} = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}} & (6)\end{matrix}$

From Expression (6), the following Expression holds. $\begin{matrix}{\lbrack {{Expression}\quad 7} \rbrack\quad{{1 - {w(t)}} = \frac{\alpha_{x{(t)}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}} & (7)\end{matrix}$

Using Expressions (6) and (7), the output signals y(t) in Expression (4)can be represented by a weighted average from the followingmultiplication and addition.

[Expression 8]y(t)=w(t)y(t-1)+(1−w(t))x(t)   (8)

Note that the weight w(t) (and 1−w(t)) used in Expression (8) can beobtained from Expression (6), from the output reliability α_(y(t-1)) forthe output signal y(t-1) from one sample back, and the input reliabilityα_(x(t)) for the current input signal x(t). Also, the output reliabilityα_(y(t)) for the current output signal y(t) in Expression (5) can alsobe obtained from the output reliability α_(y(t-1)) for the output signaly(t-1) from one sample back, and the input reliability α_(x(t)) for thecurrent input signal x(t).

Now, using the inverse numbers of the respective variances σ_(x(t)) ² orσ_(y(t)) ² as the input reliability α_(x(t)) for the input signals x(t)or the output reliability α_(y(t)) for the output signals y(t), i.e.,defining the input reliability α_(x(t)) and the output reliabilityα_(y(t)) as in the following Expression, $\begin{matrix}{\lbrack {{Expression}\quad 9} \rbrack\quad{\alpha_{x{(t)}} = \frac{1}{\sigma_{x{(t)}}^{2}}}{\alpha_{y{(t)}} = \frac{1}{\sigma_{y{(t)}}^{2}}}} & (9)\end{matrix}$the weight w(t) in Expression (6) and the weight 1−w(t) in Expression(7) can be obtained from the following Expressions. $\begin{matrix}{\lbrack {{Expression}\quad 10} \rbrack\quad{{w(t)} = \frac{\sigma_{x{(t)}}^{2}}{\sigma_{y{({t - 1})}}^{2} + \sigma_{x{(t)}}^{2}}}} & (10) \\{\lbrack {{Expression}\quad 11} \rbrack\quad{{1 - {w(t)}} = \frac{\sigma_{y{({t - 1})}}^{2}}{\sigma_{y{({t - 1})}}^{2} + \sigma_{x{(t)}}^{2}}}} & (11)\end{matrix}$

Further, σ_(y(t)) ² can be obtained by the following Expression.

[Expression 12]σ_(y(t)) ² =w(t)²σ_(y(t-1)) ²+(1−w(t))² σx(t)²   (12)

The NR circuit shown in FIG. 4 basically follows Expression (6) toperform correction parameter computing processing for computingcorrection parameters serving as the weight w(t), and uses the weightw(t) to calculate the weighting average between the output signal y(t-1)from one sample back and the current input signal x(t) followingExpression (8), thereby performing correcting processing for effectivelyremoving noise contained in the input signals x(t).

Now, the user will not necessarily feel that the output signals obtainedas the result of correction processing of the input signals by theweight w(t) obtained following Expression (6) are optimal. Accordingly,the NR circuit shown in FIG. 4 performs control data learning processingfor obtaining parameter control data for controlling (correcting) theweight w(t) serving as the correction parameter, by learning theoperations of operating unit 2 by the user, and performs correctionprocessing of input signals using weighting corrected by the parametercontrol data.

The control data learning processing is performed as follows.

That is, a weight W_(i) corresponding to learning operating signalsprovided at the i'th time by the user operating the operating unit 2 canbe considered to be that which the user thinks to be optimal with regardto the input signals input at the time that the learning operatingsignals are provided, and accordingly, obtaining parameter control datawhich can correct the weight w(t) obtained following Expression (6) to avalue near the weight W_(i) corresponding to the learning operatingsignals (ideally, the same value) is sufficient for the control datalearning processing.

To this end, let us now consider taking the weight w(t) obtainedfollowing Expression (6) as student data serving as a student forlearning, and the weight W_(i) corresponding to the learning operatingsignals as tutor data serving as a tutor for learning, and obtaining,from the weight w(t) serving as student data, a prediction value W_(i)′for the weight W_(i) serving as tutor data predicted by a linearexpression defined by a parameter control data a and b such as shown inthe following Expression.

[Expression 13]W _(i) ′=aw _(i) +b   (13)

Note that in Expression (13) (as with the later-described Expression(14) and Expressions (16) through (21)), w_(i) represents the weightw(t) serving as student data obtained following Expression (6) as toinput signals input at the time that the weight W_(i) corresponding tothe learning operating signals serving as tutor data is input.

From Expression (13), the error (prediction error) e_(i) between W_(i)serving as the tutor data and the prediction value W_(i)′ thereof isrepresented as shown in the following Expression. $\begin{matrix}{\lbrack {{Expression}\quad 14} \rbrack\quad{e_{i} = {{W_{i} - W_{i}^{\prime}}\quad = {W_{i} - ( {{aw}_{i} + b} )}}}} & (14)\end{matrix}$

Now, let us consider obtaining parameter control data a and b whichminimizes the sum of the square (second power) error of the predictionerror e_(i) in Expression (14) as represented in the followingExpression, $\begin{matrix}{\lbrack {{Expression}\quad 15} \rbrack\quad{\sum\limits_{i = 1}^{N}e_{i}^{2}}} & (15)\end{matrix}$using the least-square method (also called the least-square errormethod). Note that in Expression (15) (as with the later-describedExpressions (16) through (21)), N represents the number of sets of tutordata and student data.

First, partial differentiation of the sum of the square error inExpression (15) with each of the parameter control data a and b yieldsthe following expressions. $\begin{matrix}{{\lbrack {{Expression}\quad 16} \rbrack\quad{\frac{\partial{\sum\limits_{i = 1}^{N}\quad e_{i}^{2}}}{\partial a} = {{- 2}{\sum\limits_{i = 1}^{N}\quad{w_{i}( {W_{i} - ( {{aw}_{i} + b} )} )}}}}}\quad} & (16) \\{{\lbrack {{Expression}\quad 17} \rbrack\quad{\frac{\partial{\sum\limits_{i = 1}^{N}\quad e_{i}^{2}}}{\partial b} = {{- 2}{\sum\limits_{i = 1}^{N}\quad( {W_{i} - ( {{aw}_{i} + b} )} )}}}}\quad} & (17)\end{matrix}$

Since the minimum value (minimal value) of the sum of the square errorin Expression (15) can be obtained by a and b which make the right-handside in Expressions (16) and (17) to be 0, setting the right-hand sidein both Expressions (16) and (17) to 0 yields Expression (18) fromExpression (16), and Expression (19) from Expression (17), respectively.$\begin{matrix}{{\lbrack {{Expression}\quad 18} \rbrack\quad{{N{\sum\limits_{i = 1}^{N}{w_{i}W_{i}}}}\quad = {{{Nb}{\sum\limits_{i = 1}^{N}\quad w_{i}}} + {{Na}{\sum\limits_{i = 1}^{N}w_{i}^{2}}}}}}\quad} & (18) \\{{\lbrack {{Expression}\quad 19} \rbrack\quad{{Nb} = {{\sum\limits_{i = 1}^{N}w_{i}} - {a{\sum\limits_{i = 1}^{N}w_{i}}}}}}\quad} & (19)\end{matrix}$

Substituting Expression (19) into Expression (18) allows the parametercontrol data a to be obtained from the following Expression.$\begin{matrix}{{\lbrack {{Expression}\quad 20} \rbrack\quad{a = \frac{{N{\sum\limits_{i = 1}^{N}\quad{w_{i}W_{i}}}} - {\underset{i = 1}{\overset{N}{\sum\quad}}{w_{i}{\sum\limits_{i = 1}^{N}W_{i}}}}}{{N{\sum\limits_{i = 1}^{N}w_{i}^{2}}} - ( {\sum\limits_{i = 1}^{N}w_{i}} )^{2}}}}\quad} & (20)\end{matrix}$

Also, from Expressions (19) and (20), the parameter control data b canbe obtained from the following Expression. $\begin{matrix}{{\lbrack {{Expression}{\quad\quad}21} \rbrack\quad{b = \frac{{\sum\limits_{i = 1}^{N}W_{i}} - {a{\sum\limits_{i = 1}^{N}w_{i}}}}{N}}}\quad} & (21)\end{matrix}$

With the NR circuit shown in FIG. 4, control data learning processing isperformed for obtaining parameter control data a and b, as describedabove.

Next, the correction processing, correction parameter computingprocessing, and control data learning processing, which the NR circuitshown in FIG. 4 performs, will be described with reference to theflowcharts in FIG. 6 through FIG. 8.

First, correction processing will be described with reference to theflowchart in FIG. 6.

Upon weight w(t) serving as correction parameters being supplied fromthe selecting unit 41 of the learning unit 22 to the correcting unit 21,the weight memory 31 of the correcting unit 21 stores the weight w(t) inthe form of overwriting. Further, the computing unit 33 of thecorrecting unit 21 subtracts the weight w(t) from 1.0, obtains theweight 1−w(t), supplies this to the weight memory 32, and stores this inthe form of overwriting.

Then, upon input signals x(t) being supplied, in step S21 the computingunit 34 computes the product of the input signals x(t) and the weight1−w(t) stored in the weight memory 32, and supplies this to thecomputing unit 36. Further, in step S21, the computing unit 35 computesthe product of the weight w(t) stored in the weight memory 31 and outputsignal y(t-1) from one sample back latched by the latch circuit 37, andsupplies this to the computing unit 36.

The flow then proceeds to step S22, where the computing unit 36 adds theproduct of the input signal x(t) and the weight 1−w(t), with the productof the weight w(t) and the output signal y(t-1), thereby obtaining theweighted addition value (1−w(t))x(t)+w(t)y(t-1) of the input signal x(t)and output signal y(t-1), which is output as the output signal y(t).This output signal y(t) is also supplied to the latch circuit 37, andthe latch circuit 37 stores the output signal y(t) in the form ofoverwriting. Subsequently, the flow returns to step S21, waits for inputsignals of the next sample to be supplied, and subsequently, the sameprocessing is repeated.

Next, correction parameter computing processing will be described withreference to the flowchart shown in FIG. 7.

With correction parameter computing processing, first, in step S31, theinput reliability calculating unit 42 obtains input reliability α_(x(t))based on dispersion of the input signals, for example.

That is, the input reliability calculating unit 42 has built-in unshownFIFO (first in first out) memory which is capable of latching, besides asample x(t) of the current input signal, several past sample is, withthe dispersion thereof being calculated using the sample x(t) of thecurrent input signal and several past samples, the inverse numberthereof being obtained as the input reliability α_(x(t)), and suppliedto the output reliability calculating unit 43 and weight calculatingunit 45. Note that there are cases wherein there is not a sufficientnumber of input signal samples necessary for calculating dispersionimmediately following starting input of the input signals, but in suchcases, a default value, for example, is output as the input reliabilityα_(x(t)).

Subsequently, the flow proceeds to step S32, and the weight calculatingunit 45 obtains the weight w(t) following Expression (6), using theinput reliability α_(x(t)) from the input reliability calculating unit42.

That is, at the timing that the input reliability ax(t) is supplied fromthe input reliability calculating unit 42 to the weight calculating unit45, the output reliability α_(y(t-)1) which the output reliabilitycalculating unit 43 has output one sample back is latched at the latchcircuit 44, and in step S32, the weight calculating unit 45 obtains theweight w(t) following Expression (6) using the input reliabilityα_(x(t)) from the input reliability calculating unit 12 and the outputreliability α_(y(t-1)) latched by the latch circuit 14. This weight w(t)is supplied to the weight correcting unit 46.

Subsequently, the flow proceeds to step S33, and the weight correctingunit 46 reads the parameter control data out from the parameter controldata memory 57, and proceeds to step S34. In step S34, the weightcorrecting unit 46 judges whether or not the parameter control data readout from the parameter control data memory 57 is data of a mode whereinthe weight w(t) is not corrected, i.e., auto mode data representing amode wherein the weight w(t) is obtained automatically, so to speak,from the input reliability and output reliability at the weightcalculating unit 45 regardless of operations by the user at theoperating unit 2 is used without change as weight W for correcting theinput signals x(t) (hereafter may be referred to as auto mode).

In step S34, in the event that judgement is made that the parametercontrol data is not auto mode data, the flow proceeds to step S35, theweight correcting unit 46 corrects the weight w(t) supplied from theweight correcting unit 45 following the linear expression in Expression(13) defined by the parameter control data a and b supplied from theparameter control data memory 57, and proceeds to step S36. At step S36,the weight correcting unit 46 supplies the post-correction weight to theselecting unit 41, and proceeds to step S37. Here, in Expression (13),w_(i) is equivalent to the weight w(t) supplied from the weightcalculating unit 45, and W_(i)′ is equivalent to the post-correctionweight W.

On the other hand, in step S34, in the event that judgement is made thatthe parameter control data is auto mode data, the flow skips step S35,proceeds to step S36, where the weight correcting unit 46 supplies theweight w(t) from the weight calculating unit 45 to the selecting unit 41without change, and the flow proceeds to step S37.

In step S37, the output reliability calculating unit 43 updates theoutput reliability. That is, the output reliability calculating unit 43adds, following Expression (5), the input reliability α_(x(t)) which theinput reliability calculating unit 42 has calculated at the immediatelyprevious step S31, and the output reliability α_(y(t-1)) from one sampleback which the latch circuit 44 has latched, thereby obtaining thecurrent output reliability α_(y(t)), which is stored in the latchcircuit 44 in the form of overwriting.

Then, the flow proceeds to step S38, and the selecting unit 41 judgeswhether or not the operating unit 2 is being operated by the user, fromthe output of the operating signal processing unit 50. In the event thatjudgement is made that the operating unit 2 is not being operated instep S38, the flow proceeds to step S39, where the selecting unit 41selects the weight supplied from the weight correcting unit 46(hereafter may be referred to as correcting weight), which is output tothe correcting unit 21, and the flow returns to step S31.

Also, in step S38, in the event that judgement is made that theoperating unit 2 is being operated, the flow proceeds to step S40, wherethe selecting unit 41 selects the weight to be output by the operatingsignal processing unit 50 according to the operation, outputs this tothe correcting unit 21, and the flow returns to step S31.

Accordingly, in the correction parameter computing processing shown inFIG. 7, in the event that the operating unit 2 is not being operated,the correcting weight is supplied to the correcting unit 21, while inthe event that the operating unit 2 is being operated, weightingcorresponding to the operating signals is supplied to the correctingunit 21. As a result, at the correcting unit 21, in the event that theoperating unit 2 is not being operated, the input signals are correctedby the correction weighting, and in the event that the operating unit 2is being operated, the input signals are corrected by a weightingcorresponding to the operating signals.

Further, in the correction parameter computing processing in FIG. 7, inthe case of auto mode, weighting to be used for correction processing isobtained only from the input reliability and output reliability,regardless of the operations of operating unit 2, and if not in the automode, weighting to be used for correction processing is obtained usingthe parameter control data obtained by learning in the control datalearning processing shown in FIG. 8, which will be described later,based on the operation of operating unit 2.

Next, the control data learning processing will be described withreference to the flowchart shown in FIG. 8.

In the control data learning processing, first, in step S41, theoperating signal processing unit 50 judges whether or not learningoperating signals have been received from the operating unit 2, and inthe event that judgement is made that these have not been received, theflow returns to step S41.

Also, in the event that judgement is made in step S41 that learningoperating signals have been received from the operating unit 2, that isto say, in the event that judgment can be made that the user hasoperated the operating unit 2 so as to obtain a desired output signal,such as in cases wherein, for example, the operating unit 2 starts to beoperated, following which, without elapsing of a first time t1 orlonger, operating is performed continuously for a second time t2 orlonger, following which the operations thereof are stopped continuouslyfor a third time t3 or longer, or in cases wherein the operating unit 2starts to be operated, following which the operations thereof arestopped continuously for a third time t3 or linger, the flow proceeds tostep S42, where the tutor data generating unit 51 generates tutor data,while the student data generating unit 52 generates student data.

That is, in the event of receiving learning operating signals, theoperating signal processing unit 50 supplies weight W corresponding tothe learning operating signals (for example, weight W corresponding tothe amount of operating of the operating unit 2 or the position of knobsor levers serving as operating unit 2) to the tutor data generating unit51 and student data generating unit 52, along with a learning flag. Uponreceiving a weight W with a learning flag, the tutor data generatingunit 51 obtains the weight W as tutor data, and supplies this to thelearning data memory 53. Also the student data generating unit 52, uponreceiving the weight with a learning flag, obtains a weight wcorresponding to the input signals at that time as student data, andsupplies this to the learning data memory 53.

Here, the weight w corresponding to input signals means weight which isobtained automatically, so to speak, from the input reliability andoutput reliability, following Expression (6), and as described above,the student data generating unit 52 calculates this weight wcorresponding to the input signals, from the input signals.

Upon receiving the tutor data W from the tutor data generating unit 51and receiving the student data w from the student data generating unit52, the learning data memory 53 stores the set of the newest tutor dataW and student data w in step S43, and proceeds to step S44.

In step S44, the parameter control data calculating unit 54 performsadding in for the least-square, with regard to the tutor data andstudent data.

That is, the parameter control data calculating unit 54 performscomputation equivalent to the multiplication (w_(i)W_(i)) and summation(Σw_(i)W_(i)) of the student data w_(i) and the tutor data W_(i),computation equivalent to the summation (Σw_(i)) of the student dataw_(i), computation equivalent to the summation (ΣW_(i)) of the tutordata W_(i), and computation equivalent to the summation of the productof the student data w_(i) with itself (Σw_(i) ²) in Expressions (20) and(21).

Now, saying that, for example, N−1 sets of tutor data and student datahave already been obtained, and that the N'th set of tutor data andstudent data has been obtained as the newest tutor data and studentdata, at this point adding in with regard to the N−1′th set of tutordata and student data has already been performed at the parametercontrol data calculating unit 54. Accordingly, with regard to the N'thset of tutor data and student data, as long as the results of the addingin which has already been performed with regard to the N−1′th set oftutor data and student data has been saved, the results of adding in theN sets of tutor data and student data can be obtained including thenewest tutor data and student data, by adding in the N'th set of tutordata and student data.

Accordingly, the parameter control data calculating unit 54 is arrangedso as to store the previous adding in results in the learninginformation memory 55 as learning information, and so as to performadding in with regard to the N'th set of tutor data and student data,using this learning information. Now, the number of sets N of the tutordata and student data used for the adding in so far is also necessaryfor the adding in, and the learning information memory 55 is arranged soas to also store the number of sets N as learning information.

Following performing the adding-in in step S44, the parameter controldata calculating unit 54 stores the results of adding in, in the form ofoverwriting in the learning information memory 55 as learninginformation, and the flow proceeds to step S45.

In step S45, the parameter control data calculating unit 54 judgeswhether or not it is possible to obtain the parameter control data a andb with the Expressions (20) and (21), from the results of adding inserving as learning information stored in the learning informationmemory 55.

That is, with the set of tutor data and student data hereafter taken asa learning pair where appropriate, the parameter control data a and bcannot be obtained from the Expressions (20) and (21) unless learninginformation obtained by at least two learning pairs exists. Accordingly,in step S45, judgement is made regarding whether or not it is possibleto obtain the parameter control data a and b from the learninginformation.

In the event that judgement is made in step S45 that obtaining theparameter control data a and b is not possible, the parameter controldata calculating unit 54 supplies notification to that effect to thejudgment control unit 56, and the flow proceeds to step S49. In stepS49, the judgment control unit 56 supplies auto mode data indicating theauto mode to the parameter control data memory 57, where it is stored asparameter control data. The flow then returns to step S41, andsubsequently, the same processing is repeated.

Accordingly, in the event that learning information sufficient forobtaining the parameter control data a and b does not exist, the weightw(t) automatically obtained from the input reliability and outputreliability is used without change for correction of the input signalsx(t), as described with reference to FIG. 7.

On the other hand, in the event that judgement is made in step S45 thatthe parameter control data a and b can be obtained, the flow proceeds tostep S46, where the parameter control data calculating unit 54 obtainsthe parameter control data a and b by calculating the Expressions (20)and (21) using the learning information, which is supplied to thejudgment control unit 56, and the flow proceeds to step S47.

In step S47, the judgment control unit 56 follows the linear expressionin Expression (13) defined by the parameter control data a and b fromthe parameter control data calculating unit 54 to obtain, from eachpiece of student data stored in the learning data memory 53, predictionvalues of corresponding tutor data, and the sum of the square errorindicated in Expression (15) for the prediction error of the predictionvalue thereof (the error as to the tutor data stored in the learningdata memory 53), is obtained. Further, the judgment control unit 56obtains a normalization error wherein the sum of the square error isdivided by the number of learning pairs stored in the learning datamemory 53, for example, and the flow proceeds to step S48.

In step S48, the judgment control unit 56 judges whether or not thenormalization error is greater than (equal to or more than) apredetermined threshold value S1. In step S48, in the event that thenormalization error is judged to be greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression inExpression (13) defined by the parameter control data a and b does notapproximate the relation between the student data and tutor data storedin the learning data memory 53 in a precise manner, the flow proceeds tostep S49, where the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored, as described above. The flowthen returns to step S41, and the same processing is repeated.

Accordingly, even if the parameter control data a and b can be obtained,in the event that the linear expression in Expression (13) defined bythe parameter control data a and b does not approximate the relationbetween the student data and tutor data stored in the learning datamemory 53 in a precise manner, the weight w(t) automatically obtainedfrom the input reliability and output reliability is used without changefor correction of the input signals x(t), in the same manner as withcases wherein learning information sufficient for obtaining theparameter control data a and b does not exist.

On the other hand, in step S48, in the event that judgement is made thatthe normalization error is not greater than the predetermined thresholdvalue S1, i.e., in the event that the linear expression in Expression(13) defined by the parameter control data a and b approximates therelation between the student-data and tutor data stored in the learningdata memory 53 in a precise manner, the flow proceeds to step S50, wherethe judgment control unit 56 obtains the error (distance) ε between theregression line represented by the linear expression in Expression (13)defined by the parameter control data a and b from the parameter controldata calculating unit 54, and the point stipulated by the newest tutordata and student data stored in the learning data memory 53.

The flow then proceeds to step S51, where the judgment control unit 56judges whether or not the magnitude of the error ε is greater than(equal to or more than) the predetermined threshold value S2, and in theevent that judgement is made that it is not greater, step S52 isskipped, the flow proceeds to step S53, and the judgment control unit 56outputs the parameter control data a and b obtained in step S46 to theparameter control unit data memory 37. The parameter control data memory57 stores the parameter control data a and b from the judgment controlunit 56 in the form of overwriting, and the flow returns to step S41.

On the other hand, in the event that judgement is made in step S51 thatthe magnitude of the error E is greater than the predetermined thresholdvalue S2, the flow proceeds to step S52, where the judgment control unit56 recalculates the parameter control data a and b using only apredetermined number of learning pairs back from the newest learningpair serving as the newest tutor data and student data stored in thelearning data memory 53, by controlling the parameter control datacalculating unit 54 (without using the learning information in thelearning information memory 55). The flow then proceeds to step S53,where the judgment control unit 56 outputs the parameter control data aand b obtained in step S52 to the parameter control unit data memory 37,where it is stored in the form of overwriting, and the flow returns tostep S41.

Accordingly, in the event that the parameter control data a and b can beobtained and also in the event that the linear expression in Expression(13) defined by the parameter control data a and b approximates therelation between the student data and tutor data stored in the learningdata memory 53 in a precise manner, the weight w(t) obtained from theinput reliability and output reliability following Expression (13)defined by the parameter control data a and b obtained by learning beingperformed using the learning pairs obtained based on operation ofoperating unit 2 by the user is corrected, and corrected weight Wobtained by the correction thereof is used for correction of the inputsignals x(t).

Now, the regression line represented by the linear expression inExpression (13) defined by the parameter control data a and b obtainedin step S46 is a line which minimizes (the sum of) the square error withthe N points stipulated by the N sets of tutor data and student data asshown in FIG. 9A, and in step S50, the error E between this line and thepoint stipulated by the newest tutor data and student data, is obtained.

In the event that the magnitude of this error ε is not greater than thethreshold value S2, the regression line represented by the linearexpression in Expression (13) defined by the parameter control data aand b obtained in step S46 can be thought to be approximating each ofthe points stipulated by tutor data and student data provided so far,including the point stipulated with the newest tutor data and studentdata, in a relatively precise manner.

However, in the event that the magnitude of the error ε is greater thanthe threshold value S2, i.e., in the event that, as shown in FIG. 9B,the point stipulated by the newest tutor data and student data(indicated by a circle in FIG. 9B) greatly deviates from the regressionline represented by the linear expression in Expression (13) defined bythe parameter control data a and b obtained in step S46, it can beconceived that the user has performed operations of the operating unit 2with a tendency that differs from those so far, for some reason.

Accordingly, in this case, the judging control unit 56 recalculates theparameter control data a and b in step S52 using only several of thenewest learning pairs of the learning pairs that are stored in thelearning data memory 53, by controlling the parameter control datacalculating unit 54.

That is to say, in this case, the parameter control data calculatingunit 54 recalculates the parameter control data a and b defining theline in Expression (13) which most closely approximates the group ofpoints stipulated by a tutor data and student data, using only severalrecent tutor data and student data sets, without using (forgetting) thelearning information serving as past results of adding in that arestored in the learning information memory 55.

Specifically, as shown in FIG. 9C, for example, the parameter controldata calculating unit 54 obtains parameter control data a′ and b′defining a line which passes through a point stipulated by the newesttutor data and student data (indicated by a circle in FIG. 9C), and apoint stipulated by the tutor data and student data provided one timeback (indicated by triangle in FIG. 9C).

As described above, judgment is made regarding whether or not operatingsignals supplied according to user operations can be used for learningor not, and in the event that the signals are learning operating signalswhich can be used for learning, parameter control data a and b forcorrecting the weight for correcting input signals is learned based onthe learning operating signals, so the operations of the user can belearned without the user knowing, and as a result, processing which issuitable for the user gradually comes to be performed based on thelearned results, ultimately performing processing of which is optimalfor the user.

This means that, from the perspective of the user, after normallyoperating the operating unit 2 for a while, noise reduction resultswhich are optimal for the user can be obtained with regard to varioustypes of input signals even without making operations, and accordingly,the device becomes broken in, so to speak. At this stage of breaking inthe device, the user operates the operating unit 2 so as to obtaindesired output signals, so the relation between the operations of theoperating unit 2 and the weight W used for correcting input signalsgradually comes to be clearly recognized by the user, so ultimately, theoperations of the operating unit 2 by the user and the weight W used forcorrecting input signals come to be qualitatively correlated.

Also, at the NR circuit shown in FIG. 4, the weight W used for thecorrection processing (FIG. 6) performed at the correcting unit 21following the operations of operating unit 2 by the user is changed sothat the desired output signals by the user can be obtained. That is,upon the user operating the operating unit 2, the operating signalprocessing unit 50 outputs a weight representing the operating signalscorresponding to the operations, the selecting unit 41 selects theweight, and supplies this to the correcting unit 21. In this case, atthe correcting unit 21, correction processing indicated by Expression(8) is performed using the weight corresponding to the operations madeby the user. In the event that the weight w(t) in Expression (8) is tobe changed, due to operations made by the user, the contents of theprocessing (correction processing) represented by Expression (8) arealso changed, as a matter of course, so it can be said he that at the NRcircuit shown in FIG. 4, the “contents of processing” are being changedfollowing the operations of the user so that output signals desired bythe user can be obtained.

Further, at the NR circuit shown in FIG. 4, in the event that theparameter control data a and b cannot be obtained, or in the event thatthis can be obtained but the linear expression in Expression (13)defined by the parameter control data a and b does not approximate therelation between the student data and tutor data stored in the learningdata memory 53 in a precise manner, weighting automatically obtainedfrom the input reliability and output reliability are used forcorrection processing at the correcting unit 21. On the other hand, inthe event that the parameter control data a and b can be obtained, andalso, the linear expression in Expression (13) defined by the parametercontrol data a and b approximates the relation between the student dataand tutor data stored in the learning data memory 53 in a precisemanner, the weighting obtained from the input reliability and outputreliability is corrected following Expression (13) defined by theparameter control data a and b obtained by learning being performedusing learning pairs obtained based on operations of operating unit 2 bythe user, and corrected weighting obtained by this correcting is usedfor correction processing by the correcting unit 21.

That is to say, with the NR circuit shown in FIG. 4, in the event thatsufficient learning pairs have not been input from the user, or in theevent that learning pairs whereby highly precise approximation ispossible have not been input, weight automatically obtained from theinput reliability and output reliability is used for the correctionprocessing at the correcting unit 21, and in the event that learningpairs whereby highly precise approximation is possible have been input,correction weighting obtained by the parameter control data a and bobtained by learning being performed using the learning pairs is usedfor correction processing at the correcting unit 21.

Accordingly, the weight w(t) in Expression (8) does, after all, changebetween cases wherein a sufficient number of learning pairs or learningpairs enabling highly precise approximation are not obtained, and caseswherein learning pairs enabling highly precise approximation areobtained, and consequently, the contents of correction processingrepresented by the Expression(8) also change, so from this perspectiveas well with the NR circuit shown in FIG. 4, it can be the that the“contents of processing” are changed so that output signals desired bythe user are obtained.

Further, with the NR circuit in FIG. 4, the system for calculatingweight used for the correction processing changes depending on the caseswherein a sufficient number of learning pairs or learning pairs enablinghighly precise approximation are not obtained, and cases whereinlearning pairs enabling highly precise approximation are obtained.

That is to say, in cases wherein a sufficient number of learning pairsor learning pairs enabling highly precise approximation are notobtained, the weight is obtained from the input reliability and outputreliability, regardless of operations made by the user. On the otherhand, in the event that learning pairs enabling highly preciseapproximation has been obtained, the weighting is obtained based on theparameter control data obtained by learning using the learning pairsobtained based on the operations of the user.

Accordingly, in this case, it can be said that the processing system forcalculating weighting, i.e., the algorithm for how to obtain weighting,is changed such that output signals desired by the user can be obtained,following operations of the user.

Now, representing the processing for obtaining weight with a function F,the above-described change of “contents of processing” is equivalent tothe function F being changed. Here, cases of the function F beingchanged are generally divided into cases wherein the form of thefunction F itself changes (e.g., cases wherein F=x changes to F=x²,etc.), and cases wherein the form of the function F itself does notchange but the coefficients defining the function F change (e.g., F=2xchanges to F=3x, etc.).

Now, of the changes in “contents of processing”, if we say that caseswherein the form of the function F representing the processing of itselfchanges are changes of the “structure of processing”, changing theprocessing system for calculating weight, i.e., the algorithm for how toobtain weight, as described above, can be said to be changing the“structure of processing”.

Accordingly, with the NR circuit shown in FIG. 4, the “contents ofprocessing”, and further, the “structure of processing” is also changedfollowing operations made by the user, whereby output signals desired bythe user can be obtained.

Also, as for input signals, image signals and audio signals may be used,of course, and other signals as well. However, in the case that theinput signals are image signals, the input reliability is calculatedbased on dispersion obtained from multiple pixels near a pixel to beprocessed, either space-wise or time-wise.

Also, with the above-described case, the weight w obtained from inputreliability and the like is corrected to the correcting weight W by thelinear expression in Expression (13) defined by the parameter controldata a and b in the learning unit 22, in order to facilitatedescription, but actually, correction of the weight w is preferablyperformed by a higher expression. Also, the degree in the higherexpression is preferably set to a suitable value based on applicationsand the like to which the optimizing device 1 is applied, for example.

Further, an arrangement may be made for the expression for obtaining thecorrection weight W from the weight w (hereafter may be referred to ascorrection weight calculating expression) wherein multiple expressionsare prepared besides the linear expression W=aw+b in Expression (13),such as, for example, the quadratic expression W=aw²+bw+c, the cubicexpression W=aw³+bw²+cw+d, and so forth (wherein a, b, c, and d, arepredetermined coefficients), and wherein that of the multiple correctionweight calculating expressions which shields the smallest normalizationerror is employed. Note that in this case, the correction weightcalculating expression wherein the normalization error obtained from thelearning pair obtained by operations of the user is smallest, isselected, and the correction weighting is to be obtained using theselected correcting weight calculating expression. That is to say, thealgorithm for how to obtain the correction weight is changed followingthe operations of the user. Accordingly, in this case as well, it can besaid that the “structure of processing” is being changed followingoperations of the user.

Next, FIG. 10 illustrates another detailed configuration example of acase wherein the processing unit 11 of the optimizing device 1 shown inFIG. 4 is applied to an NR circuit. In the figure, the partscorresponding to the case in FIG. 4 are denoted with the same referencenumerals, and further description thereof will be omitted as suitable.That is to say, the NR circuit shown in FIG. 10 is basically the sameconfiguration as that shown in FIG. 4, other than not being providedwith the weight correcting unit 46, and an input reliability-calculatingunit 61 and student data generating unit 62 being provided instead ofthe input reliability calculating unit 42 and student data generatingunit 52, respectively.

The input reliability calculating unit 61 calculates the inputreliability of the input signals from multiple samples of input signalsand parameter control data stored in the parameter control data memory57, and supplies this to the output reliability calculating unit 43 andweight calculating unit 45.

The student data generating unit 62 obtains the input signals, and theoutput reliability output from the output reliability calculating unit43, as student data, and supplies this to the learning data memory 53.

Note that with the embodiment shown in FIG. 10, there is no weightcorrecting unit 46 provided, so the weight obtained in the weightcalculating unit 45 is applied to the selecting unit 41 with no change,and this selecting unit 41 is arranged so as to select and output one orthe other of the weight output by this weight calculating unit 45 andthe weight output by the operating signal processing unit 50, in thesame way as with that shown in FIG. 4.

Also, with the embodiment shown in FIG. 10, the parameter control datafunctions as data to control input reliability.

As with the NR circuit shown in FIG. 4, the NR circuit in FIG. 10 alsoperforms correction processing, correction parameter computingprocessing, and control data learning processing. The correctionprocessing is performed in the same way as the processing described withreference to FIG. 6, so description of the correction processing will beomitted for the NR circuit shown in FIG. 10, and description will bemade regarding the correction parameter computing processing and controldata learning processing.

That is to say, with the NR circuit in FIG. 10, correction parametercomputing processing and control data learning processing is performedwith the input reliability α_(x(t)) stipulating the weight shown inExpression (6) which is used in correction processing as being definedby the following expression, for example.

[Expression 22]α_(x(t)) =a _(i) x _(i) +a ₂ x ₂ + . . . +a _(N) x _(N)   (22)

Note that, however, in Expression (22), a₁, a₂, . . . , a_(N) representparameter control data, and x₁, x₂, . . . , x_(N) represent samples ofinput signals in a predetermined relation with samples of input signalswhich are to be subjected to processing (samples of interest). Now, inthe event that the input signals are image signals for example, a pixelserving as a sample of interest (indicated by an X in FIG. 11) and apixel at a position near that pixel space-wise or time-wise (indicatedby a circle in FIG. 11), can be used for x₁, x₂, . . . , x_(N), forexample.

From Expression (22), the weight w(t) provided in Expression (6) can berepresented as in Expression (23). $\begin{matrix}{{\lbrack {{Expression}\quad 23} \rbrack\quad{{w(t)} = {\frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + a_{x{(t)}}} = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + ( {{a_{1}x_{1}} + {a_{2}x_{2}} + \ldots + {a_{N}x_{N}}} )}}}}\quad} & (23)\end{matrix}$

Accordingly, in the event that the input signals x₁, x₂, . . . , x_(N),are input, parameter control data a₁, a₂, . . . , a_(N) which satisfiesthe following Expression should be obtained from Expression (23) inorder to obtain the weight W provided from the user. $\begin{matrix}{{\lbrack {{Expression}\quad 24} \rbrack\quad\text{}{W = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + ( {{a_{1}x_{1}} + {a_{2}x_{2}} + \ldots + {a_{N}x_{N}}} )}}}\quad} & (24)\end{matrix}$

As a variant of the Expression (24), Expression (25) can be obtained.

[Expression 25](a ₁ x ₁ +a ₂ x ₂ + . . . +a _(N) x _(N))W+(W−1)α_(y(t-1))=0   (25)

Obtaining the parameter control data a₁, a₂, . . . , a_(N) whichsatisfies Expression (25) at all times is generally difficult, so here,we will consider obtaining parameter control data a₁, a₂, . . . , a_(N)wherein, for example, the sum of the square error of the left-hand sideand right-hand side of the Expression (25) is smallest, by theleast-square method.

Here, making the sum of the square error of the left-hand side andright-hand side-of the Expression (25) to be smallest means making thesquare error of the weight w(t) provided by the Expression (23) and theweight W provided by the user to be smallest, i.e., with the weight Wprovided by the user as tutor data, and the input signals x₁, x₂, . . ., x_(N), and output reliability α_(y(t-1)) defining the weight w(t) inExpression (23) as student data, equivalent to making the square errorof the weight w(t) calculated from the student data by Expression (23)and the weight W serving as tutor data provided by the user to besmallest, and the weight w(t) calculated from such parameter controldata a₁, a₂, . . . , a_(N) and the student data by Expression (23) haslittle error with the tutor data W.

The square-error e² of the left-hand side and right-hand side ofExpression (25) is provided by Expression (26).

[Expression 26]e ²={(a ₁ x ₁ +a ₂ x ₂ + . . . +a _(N) x _(N))W+(W−1)α_(y(t-1))}²   (26)

The parameter control data a₁, a₂, . . . , a_(N) which makes thesquare-error e² smallest is provided by the conditions that partialdifferentiation of the square-error e² in Expression (26) by each of theparameter control data a₁, a₂, . . . , a_(N) yields 0, i.e., by thefollowing Expression. $\begin{matrix}{{\lbrack {{Expression}\quad 27} \rbrack\quad{{\frac{\partial e^{2}}{\partial a_{1}} = 0},{\frac{\partial e^{2}}{\partial a_{2}} = 0},\ldots\quad,{\frac{\partial e^{2}}{a_{N}} = 0}}}\quad} & (27)\end{matrix}$

Substituting Expression (26) into Expression (27) yields Expression(28). $\begin{matrix}{{\lbrack {{Expression}\quad 28} \rbrack\quad{{( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \ldots + {a_{N}x_{N}W} + {( {W - 1} )\alpha_{y{({t - 1})}}}} ){Wx}_{1}} = 0}\quad{{( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \ldots + {a_{N}x_{N}W} + {( {W - 1} )\alpha_{y{({t - 1})}}}} ){Wx}_{2}} = 0}\quad{{{\vdots\quad( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \ldots + {a_{N}x_{N}W} + {( {W - 1} )\alpha_{y{({t - 1})}}}} )}{Wx}_{N}} = 0}}\quad} & (28)\end{matrix}$

Accordingly, defining the matrixes X, A, and Y each as shown inExpression (29), the relation shown in Expression (30) holds for thematrixes X, A, and Y, from the Expression (28). $\begin{matrix}{{\lbrack {{Expression}{\quad\quad}29} \rbrack\quad{\begin{matrix}{X = \begin{bmatrix}{\sum\limits^{\quad}\quad{W^{2}x_{1}^{2}}} & {\sum\limits^{\quad}\quad{W^{2}x_{2}x_{1}}} & \cdots & {\sum\limits^{\quad}\quad{W^{2}x_{N}x_{1}}} \\{\sum\limits^{\quad}\quad{W^{2}x_{1}x_{2}}} & {\sum\limits^{\quad}\quad{W^{2}x_{2}^{2}}} & \cdots & {\sum\limits^{\quad}\quad{W^{2}x_{N}x_{2}}} \\\vdots & \vdots & ⋰ & \vdots \\{\sum\limits^{\quad}\quad{W^{2}x_{1}x_{N}}} & {\sum\limits^{\quad}\quad{W^{2}x_{2}x_{N}}} & \quad & {\sum\limits^{\quad}\quad{W^{2}x_{N}^{2}}}\end{bmatrix}} \\{Y = \begin{bmatrix}\begin{matrix}{\sum\limits^{\quad}\quad{{W( {1 - W} )}\alpha_{y{({t - 1})}}x_{1}}} & \quad\end{matrix} \\{\sum\limits^{\quad}\quad{{W( {1 - W} )}\alpha_{y{({t - 1})}}x_{2}}} \\\vdots \\{\sum\limits^{\quad}\quad{{W( {1 - W} )}\alpha_{y{({t - 1})}}x_{N}}}\end{bmatrix}}\end{matrix},{A = \begin{bmatrix}a_{1} \\a_{2} \\\vdots \\a_{N}\end{bmatrix}},}}\quad} & (29)\end{matrix}$[Expression 30]XA=Y   (30)

Note that the summation (Σ) in Expression (29) means the summation withregard to the number of sets, of input signals x₁ through x_(N) and theweight W provided from the user upon the input signals x₁ through x_(N)being input from the user.

Expression (30) can be solved with regard to the matrix (vector) A,i.e., the parameter control data a₁, a₂, . . . , a_(N), by the Choleskymethod or the like, for example.

The NR circuit in FIG. 10 performs control data learning processing asdescribed above, wherein, with the weight W provided by the user astutor data, and with the input signals x₁, x₂, . . . , x_(N), and outputreliability α_(y(t-1)) defining the weight w(t) in Expression (23) asstudent data, parameter control data a₁, a₂, . . . , a_(N), minimizingthe square error of the weight w(t) calculated from the student data byExpression (23) and the weight W serving as tutor data provided by theuser is learned by the least-square. Further, the NR circuit in FIG. 10obtains the input reliability α_(x(t)) from the Expression (22) definedby the parameter control data a₁ through a_(N), and further performscorrection parameter computing processing for obtaining the weightserving as correction parameters, following Expression (23), from theinput reliability α_(x(t)) and output reliability α_(y(t-1)).

Now, the correction parameter computing processing with the NR circuitshown in FIG. 10 will be described with reference to the flowchart inFIG. 12.

In the correction parameter computing processing, first, in step S61,the input reliability calculating unit 61 reads out the parametercontrol data from the parameter control data memory 57, and the flowproceeds to step S62. In step S62, the input reliability calculatingunit 61 judges whether or not the parameter control data read out fromthe parameter control data memory 57 is auto mode data which representsa mode wherein input reliability is obtained without using parametercontrol data, i.e., a mode wherein input reliability is obtainedautomatically, so to speak, from input signals alone (this mode also mayhereafter be referred to as auto mode), regardless of operations ofoperating unit 2 by the user.

In step S62, in the event that judgment is made that the parametercontrol data is not auto mode data, the flow proceeds to step S63, theinput reliability calculating unit 61 obtains using the newest N piecesof input signal samples x₁ through x_(N) supplied thereto, following thelinear expression in the Expression (22) defined by the parametercontrol data a₁ through a_(N) read out from the parameter control datamemory 57, supplies this to the output reliability calculating unit 43and weight calculating unit 45, and proceeds to step S65.

Also, in the event that the parameter control data is judged to be automode data in step S62, the flow proceeds to step S64, and the inputreliability calculating unit 61 obtains the input reliability a_(x(t))using the input signals alone, based on dispersion thereof, in the samemanner as with step S31 in FIG. 7, for example, and supplies this to theoutput reliability calculating unit 43 and weight calculating unit 45.

In step S65, the weight calculating unit 45 obtains the weight w(t)following Expression (23), using the input reliability α_(x(t)) from theinput reliability calculating unit 61, and the output reliabilityα_(y(t-1)) output one sample back by the output reliability calculatingunit 43, which is latched at the latch circuit 44. This weight w(t) issupplied from the weight calculating unit 45 to the selecting unit 41.

Subsequently, the flow proceeds to step S66, and in the same manner aswith step S37 in FIG. 7, the output reliability calculating unit 43 addsthe input reliability α_(x(t)) supplied from the input reliabilitycalculating unit 61 and the output reliability α_(y(t-1)) from onesample back, which is latched at the latch circuit 44 according toExpression (5), thereby updating the output reliability α_(y(t)), andstoring in the form of overwriting the latch circuit 44.

The flow then proceeds to step S67, where the selecting unit 41 judgeswhether or not the operating unit 2 is being operated by the user, fromthe output of operating signal processing unit 50. In the event thatjudgment is made in step S67 that operating unit 2 is not beingoperated, the flow proceeds to step S68, the selecting unit 41 selectsthe weight supplied from the weight calculating unit 45, outputs this tothe correcting unit 21, and the flow returns to step S61.

Also, in the event that judgment is made-in step S67 that the operatingunit 2 is being operated, the flow proceeds to step S69, where theselecting unit 41 selects the weight output by the output signalprocessing unit 50 according to operations thereof, outputs this to thecorrecting unit 21, and the flow returns to step S61.

Accordingly, in the correction parameter computing processing in FIG.12, in the event that the operating unit 2 is not being operated, weightcalculated based on input reliability is supplied to the correcting unit21, and in the event of operating unit 2 is being operated, weightcorresponding to the operating signals thereof are supplied to thecorrecting unit 21. Consequently, at the correcting unit 21, in theevent that the operating unit 2 is not being operated, input signals arecorrected by weighting based on input reliability, and in the event thatoperating unit 2 is being operated, input signals are corrected byweighting corresponding to the operating signals thereof.

Further, with the correction parameter computing processing in FIG. 12,in the case of the auto mode, the weight to be used for correctionprocessing is obtained from the input reliability based on dispersion ofinput signals regardless of operations of the operating unit 2, and inthe event of not being in the auto mode, the weight used for correctionprocessing is obtained from input reliability obtained using theparameter control data obtained by learning with the control datalearning processing shown in FIG. 13 which shall be described later,based on the operations of operating unit 2.

Next, the control data learning processing which the NR circuit shown inFIG. 10 performs will be described with reference to the flowchart inFIG. 13.

With the control data learning processing, first, in step S71, in thesame manner as with step S41 in FIG. 8, the operating signal processingunit 50 judges whether or not learning operating signals have beenreceived from the operating unit 2, and in the event that judgment ismade that these have not been received, the flow returns to step S71.

Also, in the event that judgment is made in step S71 that learningoperating signals have been received from the operating unit 2, i.e., inthe event that judgment can be made that the user has operated theoperating unit 2 so as to obtain a desired output signal, such as incases wherein, for example, the operating unit 2 starts to be operated,following which, without elapsing of a first time t1 or longer,operating is performed continuously for a second time t2 or longer,following which the operations thereof are stopped continuously for athird time t3 or longer, or cases wherein following starting theoperations of the operating unit 2, operations thereof are stoppedcontinuously for a third time t3 or longer, the flow proceeds to stepS72, where the tutor data generating unit 51 generates tutor data, whilethe student data generating unit 62 generates student data.

That is, in the event of receiving learning operating signals, theoperating signal processing unit 50 supplies weight W corresponding tothe learning operating signals to the tutor data generating unit 51 andstudent data generating unit 62, along with a learning flag. Uponreceiving a weight W with a learning flag, the data generating unit 51obtains the weight W as tutor data, and supplies this to the learningdata memory 53.

On the other hand, the student data generating unit 62 has a built-inbuffer (not shown) for buffering input signals, and constantly storesinput signals in the buffer to the storage capacity thereof, and in theevent of receiving a weight with the learning flag, samples x₁ throughx_(N) of input signals in a predetermined positional relation withsamples of the input signals input at that time are read out from thebuilt-in buffer. Further, the student data generating unit 62 reads outthe output reliability α_(y(t-1)) from the output reliabilitycalculating unit 43. The student data generating unit 62 then suppliesthe samples x₁ through x_(N) of input signals and output reliabilityα_(y(t-1)) to the learning data memory 53 as student data.

Upon receiving the tutor data W from the tutor data generating unit 51and the student data x₁ through x_(N) and α_(y(t-1)) from the studentdata generating unit 62, in step S73 the learning data memory 53 storesthe set (learning pair) of the newest tutor data W, and student data x₁through x_(N) and α_(y(t-1)) and the flow proceeds to step S74.

At step S74, the parameter control data calculating unit 54 performsadding-in for least-square, for the tutor data and student data.

That is, the parameter control data calculating unit 54 performscomputation of the product between one student data with another whichare components of the matrix X and Y in Expression (29), the product ofthe student data and tutor data, and the summation thereof.

Now, the adding in step S74 is performed in the same manner as that instep S44 in FIG. 8. That is to say, the results of the previousadding-in are stored in the learning information memory 55 as learninginformation, and the parameter control data calculating unit 54 usesthis learning information to perform adding in with regard to the newesttutor data and student data.

Following the adding in step S74, the parameter control data calculatingunit 54 stores the results of the adding-in within the learninginformation memory 55, as learning information, in the form ofoverwriting, the flow proceeds to step S75, and the parameter controldata computing unit 54 judges whether or not Expression (30) can besolved with regard to matrix A from the adding results as learninginformation stored in the learning information memory 55, i.e., whetheror not the parameter control data a₁ through a_(N) can be obtained.

That is to say, Expression (30) cannot be solved with regard to matrix Aunless learning information obtained from a predetermined number oflearning pairs or more exists, and the parameter control data a₁ througha_(N) serving as the components thereof cannot be obtained. Accordingly,in step S75, whether or not the parameter control data a₁ through a_(N)can be obtained from the learning information is judged.

In step S75, in the event that judgment is made that the parametercontrol data a₁ through a_(N) cannot be obtained, the parameter controldata calculating unit 54 supplies information to that effect to thejudgment control unit 56, and the flow proceeds to step S79. In stepS79, the judgment control unit 56 supplies auto mode data representingthe auto mode to the parameter control data memory 57 as parametercontrol data, where it is stored. The flow then returns to step S71, andsubsequently, the same processing is repeated.

Accordingly, in the event that sufficient learning information forobtaining the parameter control data a, through a_(N) does not exist,the weight obtained from the input reliability based on dispersion ofinput signals is used for correction of the input signals x(t), asdescribed with reference to FIG. 12.

On the other hand, in the event that judgment is made in step S75 thatthe parameter control data can be obtained, the flow proceeds to stepS76, where the parameter control data calculating unit 54 obtains theparameter control data a₁ through a_(N) by solving Expression (30) withregard to matrix A using learning information, supplies this to thejudgment control unit 56, and the flow proceeds to step S77.

At step S77, the judgment control-unit 56 follows the Expression (23)defined by the parameter control data a₁ through a_(N) from theparameter control data calculating unit 54 to obtain a prediction valueof corresponding tutor data from the student data stored in the learningdata memory 53, and obtains the sum of the square-error represented byExpression (26), of the prediction error (the error as to the tutor datastored in the learning data memory 53) of that prediction value.Further, the judgment control unit 56 obtains a normalization errorwherein the square-error sum is divided by the number of learning pairsstored in the learning data memory 53, for example, and the flowproceeds to step S78.

In step S78, the judgment control unit 56 judges whether or not thenormalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that judgment is made instep S78 that the normalization error is greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (23) defined by the parameter control data a₁ through a_(N)does not precisely approximate the relation between the student data andthe tutor data stored in the learning data memory 53, the flow proceedsto step S79, and the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored in the form of overwriting.The flow then returns to step S71, and subsequently, the same processingis repeated.

Accordingly, even in the event that the parameter control data a₁through a_(N) can be obtained, in the event that the Expression (23)defined by the parameter control data a₁ through a_(N) does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the weight obtained byinput reliability based on dispersion of input signals is used forcorrecting the input signals x(t), in the same way as cases whereinsufficient learning information for obtaining the parameter control dataa₁ through a_(N) does not exist.

On the other hand, in the event that judgment is made in step S78 thatthe normalization error is not greater than the predetermined thresholdvalue S1, i.e., in the event that the linear expression in theExpression (23) defined by the parameter control data a₁ through a_(N)does precisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S80, and the judgment control unit 56 obtains the error (distance)ε between the plane (line) of the Expression (23) defined by theparameter control data a₁ through a_(N) obtained at the parameter,control data calculating unit 54 and the point stipulated by the newesttutor data and student data stored in the learning data memory 53.

The flow then proceeds to step S81, where the judgment control unit 56judges whether or not the magnitude of the error ε is greater than(equal to or more than) a predetermined threshold value S2, and in theevent that judgment is made that this is not greater, step S82 isskipped, the flow proceeds to step S83, and the judgment control unit 56outputs the parameter control data a₁ through a_(N) obtained in step S76to the parameter control unit data memory 37. The parameter control datamemory 57 stores the parameter control data a₁ through a_(N) from thejudgment control unit 56 in the form of overwriting, the flow returns tostep S71, and subsequently, the same processing is repeated.

On the other hand, in the event that judgment is made in step S81 thatthe magnitude of the error ε is greater than the predetermined thresholdvalue S2, the flow proceeds to step S82, where the judgment control unit56 controls the parameter control data calculating unit 54 tore-calculate the parameter control data a₁ through a_(N) using only thenewest tutor data and student data stored in the learning data memory53. The flow then proceeds to step S83, where the judgment control unit56 outputs the parameter control data a₁ through a_(N) obtained in stepS82 to the parameter control data memory 57, where it is stored in theform of overwriting, and the flow returns to step S71.

That is, in the embodiment shown in FIG. 13 as well, the error ε betweenthe plane defined by Expression (23) from the parameter control data a₁through a_(N) obtained from the tutor data and student data provided sofar and the point stipulated by the newest tutor data and student datais obtained in step S82, in the same manner as with the embodiment inFIG. 8.

In the event that the magnitude of the error ε is not greater than thepredetermined threshold value S2, the plane defined by Expression (23)from the parameter control data a₁ through a_(N) obtained in step S76 isconsidered to relatively precisely approximate all of the pointsstipulated by tutor data and student data provided so far, including thepoint stipulated by the newest tutor data and student data, so thisparameter control data a₁ through a_(N) is stored in the parametercontrol data memory 57.

On the other hand, in the event that the magnitude of the error ε isgreater than the predetermined threshold value S2, the plane ofExpression (23) defined by the parameter control data al through a_(N)obtained in step S76 is considered to relatively greatly deviate fromthe points stipulated by the newest tutor data and student data, so thejudgment control unit 56 controls the parameter control data calculatingunit 54 so as to re-calculate the parameter control data a₁ througha_(N) in step S82 using only the newest tutor data and student datastored in the learning data memory 53.

With the NR circuit shown in FIG. 10, the input reliability α_(x(t)) iscalculated following Expression (22) at the input reliabilitycalculating unit 61, from the parameter control data a₁ through a_(N)obtained as described above.

Accordingly, in this case as well, learning is performed of theparameter control data a₁ through a_(N) stipulating the inputreliability α_(x(t)) in Expression (22), based on learning operatingsignals supplied according to operations of the user, so the operationsmade by the user can be learned without the user knowing, and further,optimal processing for the user can be performed using the results oflearning.

Also, with the NR circuit in FIG. 10, as with the NR circuit in FIG. 4,upon the user operating the operating unit 2, the operating signalprocessing unit 50 outputs weight representing operating signalscorresponding to the operating thereof, and the selecting unit 41selects the weight and supplies this to the correcting unit 21. In thiscase, the correction processing represented with the Expression (8) isperformed at the correcting unit 21, using the weight corresponding tothe user operations. In the event that the weight w(t) in Expression (8)is to be changed by user operations, the contents of the processing(correction processing) represented with Expression (8) is also changedas a matter of course, with the NR circuit in FIG. 10 as well, the “thecontents of processing” are changed following operations made by theuser so that output signals desired by the user are obtained.

Further, with the NR circuit in FIG. 10, in cases wherein the parametercontrol data a₁ through a_(N) cannot be obtained, or in cases whereinthis can be obtained but the Expression (23) defined by the parametercontrol data a₁ through a_(N) does not precisely approximate therelation between the student data and tutor data stored in the learningdata memory 53, weight obtained from input reliability based ondispersion of input signals is used for correction processing at thecorrecting unit 21. On the other hand, in the event that the parametercontrol data a₁ through a_(N) is obtained, and the Expression (23)defined by the parameter control data a₁ through a_(N) preciselyapproximates the relation between the student data and tutor data storedin the learning data memory 53, the weight obtained from the inputsignals and the parameter control data a₁ through a_(N) (rather, theinput reliability calculated to have therefrom) and output reliabilityis used for correction processing at the correcting unit 21 followingthe Expression (23) defined by the parameter control data a₁ througha_(N) obtained by learning being performed using learning pairs obtainedbased on operations of operating unit 2 by the user.

That is, with the NR circuit in FIG. 10, as with the NR circuit in FIG.4, algorithms for calculating weight to be used for correctionprocessing change between cases wherein sufficient numbers of learningpairs or learning pairs enabling the highly precise approximation arenot obtained, and cases wherein learning pairs enabling the highlyprecise approximation are obtained.

Accordingly, with the NR circuit in FIG. 10 as well, the “contents ofprocessing”, and further, the “structure of processing”, is also changedfollowing user operations, whereby output signals desired by the userare output.

Now, in the above case, the parameter control data a, through a_(N) isobtained using the output reliability α_(y(t-1)) as student data, withthis output reliability α_(y(t-1)) being obtained from the inputreliability α_(x(t-1)), as shown in Expression (5). The inputreliability α_(x(t)) is gradually improved toward obtaining of theweight desired by the user, by the control data learning processingshown in FIG. 13 being performed, so the output reliability α_(y(t-1))is also improved along with this.

Also, with the above-described case, the output reliability is set at aknown value, and the input reliability is stipulated by the parametercontrol data a₁ through a_(N), thereby obtaining parameter control dataa₁ through a_(N) which can yield the weight desired by the user, butconversely, an arrangement may be made wherein the input reliability isset at a known value, and the output reliability is stipulated by theparameter control data a₁ through a_(N), hereby obtaining parametercontrol data a₁ through a_(N) which can yield the weight desired by theuser.

Further, an arrangement may be made wherein, for example, the outputreliability is set at a known value, the input reliability is stipulatedby the parameter control data a₁ through a_(N), whereby parametercontrol data a₁ through a_(N) yielding weight desired by the user can beobtained, the input reliability obtained by the parameter control dataa₁ through a_(N) is further made to be a known value, and the outputreliability is stipulated by parameter control data a₁′ through a_(N)′,whereby parameter control data a₁ through a_(N) yielding weight desiredby the user can be obtained, i.e., two sets of parameter control data a₁through a_(N) and a₁′ through a_(N)′ can be obtained.

Also, in the above-described case, description has been made with theinput reliability α_(x(t)) and the output reliability α_(y(t-1)) beingdefined to obtain the parameter control data a₁ through a_(N), as shownin Expression (6), but besides this, an arrangement can be made such as,for example, shown in Expression (31) wherein the weight is definedusing, besides the input reliability α_(x(t)) and the output reliabilityα_(y(t-1)), a correction term Δα of the input reliability α_(x(t)) andthe output reliability α_(y(t-1)), so as to enable obtaining theparameter control data a₁ through a_(N) and the correction term Δα.$\begin{matrix}{{\lbrack {{Expression}\quad 31} \rbrack\quad\text{}{{w(t)} = \frac{\alpha_{y{({t - 1})}} + {\Delta\alpha}}{\alpha_{y{({t - 1})}} + {\Delta\alpha} + \alpha_{x{(t)}}}}}\quad} & (31)\end{matrix}$

Further, the expression for defining the input reliability by theparameter control data is not restricted to the Expression (22).

Further, FIG. 14 illustrates a second detailed configuration example ofthe optimizing device shown in FIG. 1. The optimizing device 1 shown inFIG. 14 is configured with an internal information generating unit 71newly provided therein, while the configuration of the processing unit11 is the same as that in FIG. 2 and accordingly description thereofwill be omitted. Also, a display unit 8 is provided externally to theoptimizing device 1 with the embodiment shown in FIG. 14.

The internal information generating unit 71 reads out internalinformation of the processing unit 11, and converts this into imageinformation, which is then output and displayed (presented) to thedisplay unit 81 made up of an LCD (Liquid Crystal Display) or CRT(Cathode Ray Tube). More specifically, the display unit 81 may bearranged to display the internal information as it is, numerically, ormay make a display wherein a level gauge like display screen is set, andthe level gauge fluctuates according to values of the internalinformation. Also, the display unit 81 is not restricted to this, andmay be any other display method so long as internal information isdisplayed (presented) visually. Now, for internal information, theweight stored in the weight memory 31 and 32 of the correcting unit 21,the stored contents of the learning data memory 53 or learninginformation memory 55 of the learning unit 22, and so forth, can beemployed. Also, the internal information may be presented to the userwith presenting (providing) methods other than displaying, i.e., bysound or the like.

Next, the optimizing processing of the NR circuit shown in FIG. 14 willbe described with reference to the flowchart shown in FIG. 15. Thisprocessing is basically the same as the optimizing processing describedwith reference to the flowchart in FIG. 3, and differs in the point thatprocessing for displaying internal information from the internalinformation generating unit 71 has been added. That is to say, the sameprocessing as that in steps S1 through S11 is performed in steps S91through S101, and the flow proceeds to step S102.

In step S102, the weight W is displayed on the display unit 81. That is,the internal information generating unit 71 reads out the value of theweight W stored in the weight memory 31, for example, converts this intoimage signals which can be displayed on the display unit 81 and outputsto the display unit 81, thereby displaying (presenting) the weight W,and the flow returns to step S91.

Due to the processing described with reference to the flowchart shown inFIG. 15, the weight W serving as internal information relating to theprocessing which is actually being executed in the processing unit 11 isdisplayed (presented) to the user, and consequently, the user canoperate the operating unit 2 so as to obtain optimal output signalswhile viewing the display of the internal information. Note that, withthe internal information generating unit 71, in addition to theabove-described internal information such as weight, the parametercontrol data a and b may be read out from the parameter control datamemory 37 (FIG. 4, FIG. 10) in the learning unit 22 and is displayed,for example. Also, an arrangement may be made to generate imageinformation, as internal information, representing the weight selectedby the selecting unit 41 (FIG. 4, FIG. 10) is a weight obtained from theparameter control data a and b obtained by learning being performedusing learning pairs, or weight obtained from the input reliability andoutput reliability.

Next, FIG. 16 illustrates the configuration example of an embodiment ofan auto-driving device for an automobile, to which the optimizing device1-shown in FIG. 1 has been applied.

With the auto-driving device, the location coordinates (X, Y) of theautomobile and the direction of driving θ are obtained, and theautomobile is driven following a predetermined course. However, errorsare often contained in the location coordinates (X, Y) and the directionof driving θ obtained with the auto-driving device, and in this case,the automobile may run off of the predetermined course. Accordingly,with the auto-driving device shown in FIG. 16, the operations of theuser are learned without the user knowing, and the automobile is causedto drive along the predetermined course, based on the results oflearning. That is, in the event that the automobile runs off thepredetermined course, generally, the user will operate the steeringwheel or gas pedal to cause the automobile to run along thepredetermined course. Accordingly, the auto-driving device shown in FIG.16 effects control such that the operations of the user are learnedwithout the user knowing, and the automobile is caused to gradually cometo drive along the predetermined course, based on the results oflearning.

A gyro sensor 91 detects a yaw rate r of the automobile, and suppliesthis to a computing unit 93. A wheel pulser 92 supplies electric pulsesof a number corresponding to the rotational angle of the wheels of theautomobile, to the computing unit 93.

The computing unit 93 computes the location coordinates (X, Y) of theautomobile and the direction of driving θ from the output of the gyrosensor 91 and wheel pulser 92, following the following expression, forexample, and supplies this to the optimizing device 94. $\begin{matrix}{\lbrack {{Expression}\quad 32} \rbrack\quad\begin{matrix}{{\theta = {{\theta(0)} + {\int{r{\mathbb{d}t}}}}}\quad} \\{X = {{X(0)} + {\int{V_{r}{\cos( {\theta + \beta} )}{\mathbb{d}t}}}}} \\{Y = {{Y(0)} + {\int{V_{r}{\sin( {\theta + \beta} )}{\mathbb{d}t}}}}}\end{matrix}} & (32)\end{matrix}$

Note that in Expression (32), θ(0) represents the direction of theautomobile at the time of starting driving, and (X(0), Y(0)) representthe coordinates of the automobile at the time of starting driving. Also,θ(0) and (X(0), Y(0)) may be obtained using an unshown GPS (GlobalPositioning System) or the like. Further, V_(r) represents the drivingspeed of the automobile, and represents the slip angle of the center ofgravity of the automobile.

Now, how to obtain the coordinates (X, Y) of the automobile and drivingdirection 0 as described above is disclosed in, for example, JapaneseUnexamined Patent Application Publication No. 10-69219.

The optimizing device 94 is configured of a processing unit 101, learnsthe operations which the user makes with an operating unit 98, i.e.,perform's learning based on operating signals supplied by the useroperating the operating unit 98, corrects the coordinates (X, Y) anddriving direction θ from the computing unit 93 so that driving which theuser desires is performed, based on the results of learning, andsupplies this to the auto-driving control unit 95.

The auto-driving control unit 95 stores map data and a course which hasbeen set beforehand for auto-driving (hereafter may be referred to as aset course). The auto-driving control unit 95 recognizes the currentposition and driving direction of the automobile from the coordinates(X, Y) and driving direction θ supplied from the processing unit 101 ofthe optimizing device 94, and generates control signals for controllinga later-described driving unit 97, so that the automobile follows theset course for driving, and outputs this to the selecting unit 96.

The selecting unit 96 is supplied with control signals from theauto-driving control unit 95, and is also supplied with operatingsignals from the operating unit 98. Of the control signals from theauto-driving control unit 95 and the operating signals from theoperating unit 98. The selecting unit 96 selects the operating signalswith higher priority, and outputs these to the driving unit 97, that is,the selecting unit 96 normally selects the control signals from theauto-driving control unit 95 and outputs these to the driving unit 97,but upon receiving operating signals from the operating unit 98, stopsoutput of the control signals from the auto-driving control unit 95while operating signals are being received, and outputs the operatingsignals from the operating unit 98 to the driving unit 97.

The driving unit 97 follows control signals or operating signals fromthe selecting unit 96, and drives the mechanisms of the automobile whichare necessary for driving, such as an unshown engine, wheels, brakes,clutch, etc. The operating unit 98 is configured of, for example, thesteering wheel, gas pedal, brake pedal, clutch pedal, etc., and suppliesoperating signals corresponding to operations made by the user, to theoptimizing device 94 and selecting unit 96.

With the auto-driving device configured as described above, at thecomputing unit 93, the current coordinates (X, Y) and driving directionθ of the automobile are computed from the output of the gyro sensor 91and wheel pulser 92, and supplied to the auto-driving control unit 95via the processing unit 101 of the optimizing device 94. Theauto-driving control unit 95 recognizes the current position and drivingdirection of the automobile from the coordinates (X, Y) and drivingdirection θ supplied thereto, generates control signals for controllingthe later-described driving unit 97 so that the automobile drivesfollowing the set course, and supplies these to the driving unit 97 viathe selecting unit 96. Thus, the automobile performs auto-drivingfollowing the control signals which the auto-driving control unit 95outputs.

On the other hand, upon the user operating the operating unit 98,operating signals corresponding to the operation thereof are supplied tothe driving unit 97 via the selecting unit 96, whereby the automobiledrives following the operations which the user makes at the operatingunit 98.

Further, due to the user operating the operating unit 98, operatingsignals which operating unit 98 outputs are supplied to the processingunit 101 of the optimizing device 94 as well. The optimizing device 94performs learning based on operating signals supplied by the useroperating the operating unit 98. Upon the user stopping the operationsof the operating unit 98, the processing unit 101 of optimizing device94 then makes corrections to the coordinates (X, Y) and drivingdirection θ supplied from the computing unit 93 so that driving isperformed following the set course as the driving which the userdesires, based on the results of learning, and supplies this to theauto-driving control unit 95.

Next, FIG. 17 illustrates a configuration example of the processing unit101 of the optimizing device 94 in FIG. 16. In the figure, the partscorresponding to the processing unit 11 shown in FIG. 4 are denoted withthe same reference numerals, and description thereof will be omittedwhere suitable in the following. That is, the processing unit 101 shownin FIG. 17 is configured basically the same as the processing unit 11shown in FIG. 4 except that no selecting unit 41 is provided, and anoperating signal processing unit 110 and tutor data generating unit 111are provided instead of the operating signal processing unit 50 andtutor data generating unit 51, respectively.

Now, to facilitate description in the following, of the coordinates (X,Y) and driving direction θ supplied from the computing unit 93 to theprocessing unit 101 of the optimizing device 94, description will bemade giving consideration to the driving direction θ alone. However, thecoordinates (X, Y) may be subjected to the same processing as thedriving direction θ to be described now.

The operating signal processing unit 110 receives operating signals fromthe operating unit 98, and judges whether or not these are learningoperating signals. In the event the operating signals are learningoperating signals, the operating signal processing unit 110 thensupplies a message to that effect to the student data generating unit 52and tutor data generating unit 111.

The tutor data generating unit 111 is arranged so as to be supplied withmessages from the operating signal processing unit 110 to the effectthat the operating signals are learning operating signals (hereafter maybe referred to as learning messages), and also so as to be supplied withthe driving direction θ from the computing unit 93, serving as inputsignals. Further, the driving direction θ from the computing unit 93which has been corrected (hereafter may be referred to as correcteddriving direction), is also supplied to the tutor data generating unit111 as output signals output from the correcting unit 21 (computingdevice 36). The tutor data generating unit 111 obtains the weight wcorresponding to the learning operating signals, from the drivingdirection θ serving as input signals supplied at the time of receivingthe learning message, and the corrected driving direction serving asoutput signals, which is supplied to the learning data memory 53 astutor data.

That is, in this case, it is necessary to obtain, as tutor data, theweight W at the point of the automobile heading in a predetermineddirection following the user operating the operating unit 98 which isthe steering wheel so that the automobile heads in a predetermineddirection. That is, it is necessary to employ as tutor data the weight Wused for correction of the input signals x(t) indicating the drivingdirection θ, immediately following the user operating the operating unit98 serving as the steering wheel and the automobile heading in thepredetermined direction. The input signals x(t) immediately followingoperation of the operating unit 98 are corrected, following Expression(8), into output signals y(t) immediately following operating theoperating unit 98 by weighted addition of the input signals x(t) and theoutput signals y(t-1) output immediately before operating the operatingunit 98, so the weight W used for correction of the input signals x(t)immediately following operation of the operating unit 98 can be obtainedfrom Expression (8), from the input signals x(t) immediately followingoperation of the operating unit 98, and output signals y(t) immediatelyfollowing operating the operating unit 98 and output signals y(t-1)output immediately before operating the operating unit 98. Accordingly,the tutor data generating unit 111 obtains the weight W from the drivingdirection θ serving as the input signals x(t) supplied immediatelyfollowing a reception of the learning message, and the corrected drivingdirection which is the output signals y(t-1) and y(t) suppliedimmediately before and immediately following reception of the learningmessage, respectively, and supplies this to the learning data memory 53.

Also, upon receiving the learning message, the student data generatingunit 52 supplies the weight w obtained from the driving direction whichis the input signals supplied to immediately before that point, to thelearning data memory 53, as student data. That is, as described withreference to FIG. 4, the student data generating unit 52 is configuredin the same way as the input reliability calculating unit 42, outputreliability calculating unit 43, latch circuit 44, and weightcalculating unit 45, and calculates the weight w of the drivingdirection which is the input signals supplied thereto (the same weight wthat is obtained at the weight calculating unit 45), and supplies theweight w calculated immediately before receiving the learning message tothe learning data memory 53, as student data.

Accordingly, at the parameter control data calculating unit 54, theweight W at the point that the user operates the operating unit 98 andthe driving direction becomes the direction which the user desires istaken as tutor data, and the same weight w as that which the weightcalculating unit 45 output immediately before the user operating theoperating unit 98 is taken as student data, and the parameter controldata a and b shown in the Expressions(20) and (21) is performed.

Then, at the weight correcting unit 46, the weight w obtained at theweight calculating unit 45 is corrected following Expression (13), usingthe parameter control data a and b, and this is supplied to thecorrecting unit 21.

As a result, the parameter control data a and b serve to correct theweight w obtained at the weight calculating unit 45 so as to correct thedriving direction immediately before the user operating the operatingunit 98 to the driving direction immediately following the useroperating the operating unit 98, so the automobile performs auto-drivingalong the set course.

That is, it can be considered that the fact that the user operates theoperating unit 98 means that the driving direction θ which the computingunit 93 outputs contains error due to error in the gyro sensor 91 ornoise in the output thereof, computation error in the computing unit 93,etc., and accordingly does not represent the true driving direction ofthe automobile, and accordingly the actual driving direction of theautomobile has deviated from the set course. Further, it can beconsidered that the operations of the operating unit 98 made by the enduser in this case are for changing the actual driving direction of theautomobile to that following the set course. Accordingly, performinglearning wherein the weight W at the point that the user operatesoperating unit 98 and the actual driving direction of the automobilefollows the set course is taken as tutor data, and the weight w obtainedat the weight calculating unit 45 immediately before the user operatesthe operating unit 98, i.e., the weight w which the weight calculatingunit 45 outputs in the state of deviating from the set course, asstudent data, the parameter control data a and b of the Expression (13)for correcting the weight in the Expression (6), so as to correct thedriving direction in the state of deviating from the set course to adirection following the set course.

Next, description will be made regarding the processing unit 101 of theoptimizing device 94 shown in FIG. 17. Now, as with the processing unit11 of the NR circuit shown in FIG. 4, the processing unit 101 of theoptimizing device 94 shown in FIG. 17 performs control data learningprocessing for obtaining the parameter control data for controlling(correcting) the weight serving as correction parameters, by correctionprocessing for correcting the driving direction θ which the computingunit 93 outputs as input signals x(t), correction parameter computingprocessing for obtaining the weight serving as correction parameter isto be used for the correction processing thereof, and operations of theoperating unit 98 (FIG. 16) by the user, but the correction processingis the same as the correction processing performed by the NR circuitshown in FIG. 4 that has been described with reference to FIG. 7, sohere, the correction parameter computing processing and student datalearning processing which the processing unit 101 of the optimizingdevice 94 shown in FIG. 17 performs, will be described.

First, the correction parameter computing processing which theprocessing unit 101 of the optimizing device 94 shown in FIG. 17performs will be described with reference to the flowchart shown in FIG.18.

In the correction parameter computing processing, first, in step Sill,the input signals calculating unit 22 obtains the input reliabilityαx(t) based on dispersion of the driving direction θ from the computingunit 93 (FIG. 16) serving as input signals, and supplies this to theoutput reliability calculating unit 43 and weight calculating unit 45,in the same way as with step S31 in FIG. 7.

Subsequently, the flow proceeds to step S112, where the weightcalculating unit 45 uses the input reliability αx(t) from the inputreliability calculating unit 42 to obtain the weight w(t) which issupplied to the weight correcting unit 46, following Expression (6), andthe flow proceeds to step S113.

In step S113, the weight correcting unit 46 reads out the parametercontrol data from the parameter control data memory 57, and the flowproceeds to step S114. In step S114, the weight correcting unit 46judges whether or not the parameter control data read out from theparameter control data memory 57 is auto mode data which represents amode wherein the weight w(t) is not corrected, i.e., a mode wherein aweight w(t) which is obtained automatically, so as to say, from theinput reliability and output reliability at the weight calculating unit45, as the weight W for correcting the input signals x(t) without anychange, regardless of operations of operating unit 98 (FIG. 16) by theuser (auto mode).

In the event that judgment is made in step S113 that the parametercontrol data is not auto mode data, the flow proceeds to step S115,where the weight correcting unit 46 corrects the weight w(t) suppliedfrom the weight calculating unit 45 following the linear expression inthe Expression (13) defined by the parameter control data a and bsupplied from the parameter control data memory 57, and the flowproceeds to step S116. In step S116, weight correcting unit 46 suppliesthe post-correction weight to the correcting unit 21 as correctionparameters, and the flow proceeds to step S117.

On the other hand, in the event that judgment is made in step S114 thatthe parameter control data is auto mode data, step S115 is skipped, theflow proceeds to step S116 where the weight correcting unit 46 suppliesthe weight w(t) from the weight calculating unit 45 without change tothe correcting unit 21 as correction parameters, and the flow proceedsto step S117.

At step S117, the output reliability calculating unit 43 updates theoutput reliability. That is, the output reliability calculating unit 43adds the input reliability α_(x(t)) calculated by input reliabilitycalculating unit 42 immediately preceding step S31 and the outputreliability α_(y(t-1)) from one sample back which the latch circuit 44has latched, following Expression (5), thereby obtaining the currentoutput reliability α_(y(t)), which is stored in the form of overwritingon the latch circuit 44.

Following the processing in step S117, the flow returns to step S111,and subsequently the same processing is repeated.

As described above, with the correction parameter computing processingshown in FIG. 18, in the event of being in a auto mode, the weight to beused for correction processing is obtained from the input reliabilityand output reliability regardless of the operations of the operatingunit 98, and in the event of not being in the auto mode, the weight tobe used for correction processing is obtained using the parametercontrol data obtained by the learning in the control data learningprocessing in FIG. 19 which will be described later, based on theoperations of the operating unit 98.

Next, description will be made regarding the control data learningprocessing which the processing unit 101 of the optimizing device 94shown in FIG. 17 performs, with reference to the flow chart shown inFIG. 19.

With the control data learning processing, first, in step S131, theoperating signal processing unit 110 judges whether or not learningoperating signals have been received from the operating unit 98 (FIG.16), and in the event that judgment is made that the signals have notbeen received, the flow returns to step S101.

Also, in the event that judgment is made in step S101 that learningoperating signals have been received from the operating unit 98, i.e.,in the event of that, for example, the steering wheel or the like whichis the operating unit 98 starts to be operated, following which, withoutelapsing of a first time t1 or longer, operating is performedcontinuously for a second time t2 or longer, following which theoperations thereof are stopped continuously for a third time t3 orlonger, or cases wherein following starting the operations of thesteering wheel which is the operating unit 98, operations thereof arestopped continuously for a third time t3 or longer, such that judgmentcan be made that the user has operated the steering wheel which isoperating unit 98 so as to direct the automobile in the desireddirection, the flow proceeds to step S132, where the tutor datagenerating unit 111 generates tutor data, and the student datagenerating unit 52 generates student data.

That is to say, upon judging that the learning operating signals havebeen received, the operating signal processing unit 110 supplies alearning message to the tutor data generating unit 111 and student datagenerating unit 52. Upon receiving the learning message from theoperating signal processing unit 110, in step S132 the tutor datagenerating unit 111 obtains a weight W corresponding to the learningoperating signals, from the driving direction θ which is the inputsignals supplied from the computing unit 93, and the corrected drivingdirection θ from the computing unit 93 (corrected driving direction)which is the output signals output by the correcting unit 21 (computingdevice 36).

Specifically, the tutor data generating unit 111 receives input signalsx(t) representing the driving direction θ immediately following the useroperating the operating unit 98 serving as the steering wheel and theautomobile heading in the desired direction, from the computing unit 93(FIG. 16). Further, the tutor data generating unit 111 is arranged so asto hold the current output signals y(t) output by the correcting unit21, and output signals y(t-1) from one timing back, i.e., the outputsignals y(t-1) immediately before operating the operating unit 98, andobtains the weight W used at the correcting unit 21 at the time that thelearning operating signals are provided to (the weight corresponding tothe learning operating signals) following Expression (8), using theinput signals x(t), and output signals y(t) and y(t-1).

Note that in this case, operations of the steering wheel which is theoperating unit 98 made by the user are considered to be instantaneouslycompleted within one timing from t-1 to t, to facilitate description.

The tutor data generating unit 111 obtains the weight W corresponding tothe learning operating signals as described above, and then supplies theweight W to the learning data memory 53 as tutor data.

Further, in step S132, the student data generating unit 52 which hasreceived the learning message from the operating signal processing unit110 supplies the same weight w as that output by the weight calculatingunit 45, calculated using the input reliability and output reliabilityobtained from the driving direction which is input signals supplied fromthe computer unit 93 (FIG. 16), to the learning data memory 53, asstudent data.

Accordingly, the learning data memory 33 is supplied with a learningpair, wherein the weight W used at the correcting unit 21 at the pointthat the actual driving direction of the automobile becomes thedirection which the user desires by the user operating the operatingunit 98, is taken as tutor data, and the weight w obtained from theinput reliability and output reliability immediately before the useroperating the operating unit 98, is taken as student data.

Upon receiving the tutor data W from the tutor data generating unit 111and the student data w from the student data generating unit 52, thelearning data memory 53 stores the set of the newest tutor data W andstudent data w in step S133, and the flow proceeds to step S134.

In step S134, the parameter control data calculating unit 54 performsadding in the least-square, with regard to the newest tutor data andstudent data stored in the learning data memory 53 and to the learninginformation stored in the learning information memory 55, in the sameway-as with step S44 in FIG. 8. Further, in step S134, the parametercontrol data calculating unit 54 stores the adding-in results aslearning information, in the form of overwriting the learninginformation memory 55, and the flow proceeds to step S135.

In step S135, the parameter control data calculating unit 54 judgeswhether or not the parameter control data a and b can be obtained withthe Expressions (20) and (21), from the adding-in results serving as thelearning information stored in the learning information memory 55, aswith the case in step S45 in FIG. 8.

In the event that judgment is made in step S135 that the parametercontrol data a and b cannot be obtained, the parameter control datacalculating unit 54 supplies the judgment control unit 56 withinformation to that effect, and the flow proceeds to step S139. In stepS139, the judgment control unit 56 supplies the auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored. Then, the flow returns tostep S131, and subsequently, the same processing is repeated.

Accordingly, in the event that sufficient learning information wherebythe parameter control data a and b can be obtained does not exist, theweight w(t) automatically obtained from the input reliability and outputreliability at the weight calculating unit 45 (FIG. 17) is used forcorrection of input signals x(t) without change.

On the other hand, in the event that judgment is made in step S135 thatthe parameter control data a and b can be obtained, the flow proceeds tostep S136, where the parameter control data calculating unit 54 useslearning information to calculate the Expressions (20) and (21), andthereby obtaining the parameter control data a and b, which is suppliedto the judgment control unit 56, and the flow proceeds to step S137.

In step S137, the judgment control unit 56 follows the linear expressionin Expression (13) defined by the parameter control data a and b fromthe parameter control data calculating unit 54 to obtain predictionvalues of a corresponding tutor data from the student data stored in thelearning data memory 53, and obtains the sum of the square-errorrepresented by the Expression (15) for the prediction error of thatprediction value (error as to the tutor data stored in the learning datamemory 53). Further, the judgment control unit 56 obtains anormalization error wherein the square-error sum is divided by thenumber of the learning pairs stored in the learning data memory 53, forexample, and the flow proceeds to step S138.

In step S138, the judgment control unit 56 judges whether or not thenormalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that judgment is made instep S138 that the normalization error is greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (13) defined by the parameter control data a and b does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S139 where the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored, as described above. The flowthen returns to step S131, and subsequently, the same processing isrepeated.

Accordingly, even in the event that the parameter control data a and bcan be obtained, in the event that the linear expression in theExpression (13) defined by the parameter control data a and b does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the weight w(t)automatically obtained from the input reliability and output reliabilityis used for correcting the input signals x(t), in the same way as caseswherein sufficient learning information for obtaining the parametercontrol data a and b does not exist.

On the other hand, in the event that judgment is made in step S138 thatthe normalization error is not greater than the predetermined thresholdvalue S1, i.e., in the event that the linear expression in theExpression (13) defined by the parameter control data a and b doesprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S140, wherein the judgment control unit 56 obtains the error(distance) ε between the regression line represented by the linearexpression in the Expression (13) defined by the parameter control dataa and b from the parameter control data calculating unit 54 and a pointstipulated by the newest tutor data and student data stored in thelearning data memory 53.

The flow then proceeds to step S141, where the judgment control unit 56judges whether or not the magnitude of the error ε is greater than(equal to or more than) a predetermined threshold value S2, and in theevent that judgment is made that this is not greater, step S142 isskipped, the flow proceeds to step S143, and the judgment control unit56 outputs the parameter control data a and b obtained in step S136 tothe parameter control data memory 37. The parameter data memory 57stores the parameter control data a and b from the judgment control unit56 in the form of overwriting, and the flow returns to step S131.

On the other hand, in the event that judgment is made in step S141 thatthe magnitude of the error ε is greater than the predetermined thresholdvalue S2, the flow proceeds to step S142, where the judgment controlunit 56 controls the parameter control data calculating unit 54 tore-calculate the parameter control data a and b using only apredetermined number of the past learning pairs of from the newestlearning pairs of newest tutor data and student data stored in thelearning data memory 53 (without using the learning information in thelearning information memory 55) by controlling the parameter controldata calculating unit 54. The flow then proceeds to step S143, where thejudgment control unit 56 outputs the parameter control data a and bobtained in step S142 to the parameter control data memory 37, where itis stored in the from of overwriting, and the flow returns to step S131.

Accordingly, in the event that the parameter control data a and b can beobtained, and linear expression of the Expression (13) defined by theparameter control data a and b precisely approximates the relationbetween the student data and tutor data stored in the learning datamemory 53, the weight w(t) obtained from the input reliability andoutput reliability is corrected following the Expression (13) defined bythe parameter control data a and b obtained by the learning beingperformed using the learning pairs obtained based on the operations ofthe operating unit 2 by the user, and the corrected weight W obtained bythe correction is used for correcting the input signals x(t).

As described above, with the auto-driving device shown in FIG. 16 aswell, judgment is made regarding whether or not the operating signalssupplied in response to operations of the user can be used for learning,and in the event that these are learning operating signals which can beused for learning, the parameter control data a and b for correcting theweight for correcting the input signals is learned based on the learningoperating signals, so the operations of the user can be learned withoutthe user knowing, and consequently, processing suitable for the usergradually comes to be performed based on the results of learning, andultimately, processing optimal for the user is performed.

That is, as the user performs operations with operating unit 98 so as tocorrect the driving direction to follow the set course, the automobilegradually comes to perform auto-driving following the set course.

Also, in the same manner as the NR circuit shown in FIG. 4, with theprocessing unit 101 of the optimizing device 94 shown in FIG. 17 aswell, the weight W used for the correction processing (FIG. 6) performedat the correcting unit 21 is changed, following the operations of theuser at the operating unit 98, so that the actual driving direction ofthe automobile follows the set course. That is to say, upon the useroperating the operating unit 98 so that the driving direction of theautomobile heads in the desired direction, the driving direction θ whichis input signals output by the computing unit 93 (FIG. 16) changes, sothe input reliability obtained from the driving direction θ, and furtherthe output reliability obtained from the input reliability, change. Thechange in this input reliability and output reliability causes theweight obtained at the weight calculating unit 45 to change, and thischanged weight is supplied to the correcting unit 21 through the weightcorrecting unit 46. The correction processing represented by Expression(8) is then performed at the correcting unit 21, using the weight thathas been supplied. Accordingly, in the event that the user operates theoperating unit 98, the weight of the Expression (8) is changed by theoperations of the user, and then the same manner as the case describedwith the NR circuit in FIG. 4, the contents of the processing(correction processing) represented by Expression (8) also change as amatter of course, so it can be said that the “contents of processing”are being changed so that the driving direction desired by the user canbe obtained, following operations of the user, with the processing unit101 of the optimizing device 94 shown in FIG. 17, as well.

Further, as with the NR circuit in FIG. 4, with the processing unit 101of the optimizing device 94 shown in FIG. 17 as well, in cases whereinsufficient learning pairs have not been input from the user, or in theevent that learning pairs whereby highly precise approximation ispossible have not been input, weight automatically obtained from theinput reliability and output reliability is used for the correctionprocessing at the correcting unit 21, and in the event that learningpairs whereby highly precise approximation is possible have been input,correction weighting obtained by the parameter control data a and bobtained by learning being performed using the learning pairs is usedfor correction processing at the correcting unit 21. That is to say, thealgorithms for calculating weight used for the correction processingchange depending on the cases wherein a sufficient number of learningpairs or learning pairs enabling highly precise approximation are notobtained, and cases wherein learning pairs enabling highly preciseapproximation are obtained.

Accordingly, as with the NR circuit in FIG. 4, with the processing unit101 of the optimizing device 94 in FIG. 17 as well, the “contents ofprocessing”, and further, the “structure of processing”, is also changedfollowing user operations, whereby the automobile performs auto-drivingin the driving direction following the set course.

Also, for example, Japanese Unexamined Patent Application PublicationNo. 7-13625 discloses that driving control device for work vehicles suchas rice transplanters and the like, and with the driving control device,the amount of correction of control parameters in the automatic steeringstate is computed so that the difference between the state of operationby the user and the information based on detection results such as ofthe gyro sensor and the like, are reduced. Accordingly, the auto drivingdevice shown in FIG. 16 holds in common with the driving control devicedescribed in Japanese Unexamined Patent Application Publication No.7-13625, the point that the amount of correction of parameters for autodriving (automatic steering) changes based on operations made by theuser.

However, the auto driving device shown in FIG. 16 greatly differs fromthe driving control device described in Japanese Unexamined PatentApplication Publication No. 7-13625 which computes the amount ofcorrection of control parameters in the auto steering state only in theevent that a switch is manually switched over to a manual steeringcontrol mode, with regard to the point of judging whether or notoperating signals supplied according to operations made by the user canbe used for learning, and the point that, in the event that theoperating signals are learning operating signals which can be used forlearning, parameter control data for correcting the weighting forcorrecting input signals is learned based on the learning operatingsignals.

Due to such differences, with the driving control device described inJapanese Unexamined Patent Application Publication No. 7-13625, everytime the user feels that suitable automatic steering is not beingperformed, the user must switch over a switch to a manual steeringcontrol mode, and following computation of the amount of correcting thecontrol parameters, switch over the switch again to the auto steeringcontrol mode, which may leave the user with an impression of beingtroublesome.

Conversely, with the auto driving device shown in FIG. 16, the judgmentis made regarding whether or not operating signals supplied according tooperations made by the user can be used for learning, and further, inthe event that these are learning operating signals which can be usedfor learning, the algorithms are changed such that parameter controldata for correcting the weighting for correcting input signals islearned based on the learning operating signals, so suitable autodriving is performed even without the user performing such switchingover of the switch, as described above. That is to say, learning ofoperations by the user is performed without the user knowing, solearning proceeds as the user corrects the driving direction, and theautomobile gradually comes to drive following the set course, evenwithout the user making operations.

Further, with the auto driving device shown in FIG. 16, the structure ofprocessing changes corresponding to user operations, thus differing fromthe driving control device described in Japanese Unexamined PatentApplication Publication No. 7-13625 with regard to this point as well.

Next, FIG. 20 illustrates another configuration example of theprocessing unit 101 of the optimizing device 94 shown in FIG. 16. Notethat the parts corresponding with the case shown in FIG. 17 in thefigure are denoted with the same reference numerals, and descriptionthereof will be omitted as appropriate in the following.

With the processing unit 11 of the NR circuit shown in FIG. 4 and FIG.10, and the processing unit 101 of the optimizing device 94 shown inFIG. 17, parameter control data for controlling correction parametersare learned using learning pairs obtained based on operations made bythe user, but with the processing unit 101 shown in FIG. 20, correctionparameters themselves are learned using the learning pairs obtainedbased on operations made by the user.

That is, with the embodiment shown in FIG. 20, the correcting unit 21 isconfigured of a correction amount calculating unit 141 and computingdevice 142, and the learning unit 22 is configured of learning datamemory 53, learning information memory 55, judgment control unit 56,operating signal processing unit 110, tutor data generating unit 143,student data generating unit 144, correction parameter calculating unit145, and correction parameter memory 146.

The later-described correction parameters are supplied to the correctionamount calculating unit 141 from the correction parameter memory 146 inthe learning unit 22, with the correction amount calculating unit 141using the correction parameters to calculate the correction amount forcorrecting the driving direction θ which is the input signals, andsupplying this to the computing device 142.

The computing device 142 is supplied with correction amount from thecorrection amount calculating unit 141, and is also supplied with thedriving direction θ which is the input signals, from the computing unit93 (FIG. 16), with the computing unit 142 correcting the drivingdirection θ which is the input signals by adding the correction amountthereto, and outputting the post-correction driving direction (correcteddriving direction) to the auto driving control unit 95 (FIG. 16) asoutput signals.

The tutor data generating unit 143 supplies the driving direction whichis the input signals supplied immediately following reception of thelearning message from the operating signal processing unit 110, to thelearning data memory 53, as tutor data. The student data generating unit144 supplies the driving direction which is the input signals suppliedimmediately before reception of the learning message from the operatingsignal processing unit 130, to the learning data memory 53, as studentdata.

The correction parameter calculating unit 145 learns correctionparameter which minimizes a predetermined statistical error by computingnew learning information, using tutor data and student data serving aslearning data stored in the learning data memory 53, and further, ifnecessary, using learning information stored in learning informationmemory 55, and supplies this to the judgment control unit 56 undercontrol of the judgment control unit 56. The correction parametercalculating unit 145 updates the stored contents of the learninginformation memory 55, with the new learning information obtained bylearning.

The correction parameter memory 146 stores correction parameters whichthe judgment control unit 56 outputs.

With optimizing device 94 configured as described above, the drivingdirection θ supplied from the computing unit 93 is corrected asdescribed next.

That is, with the yaw rate output by the gyro sensor 91 (FIG. 16) at atime t as r′, the driving direction is calculated at the computing unit93 from an expression wherein r in Expression (32) is substituted withr′.

Now, with the error contained in the yaw rate r′ output by the gyrosensor 91 as e_(r), and the true yaw rate as r, the yaw rate r′ outputby the gyro sensor 91 is represented by the following expression.

[Expression 33]r′=r+e _(r)   (33)

At the computing unit 93, the driving direction θ′ calculated from theyaw rate r′ output from the gyro sensor 91 is as follows, from theExpressions (32) and (33). $\begin{matrix}{\lbrack {{Expression}\quad 34} \rbrack\quad\begin{matrix}{{\theta^{\prime} = {{\theta(0)} + {\int{r^{\prime}{\mathbb{d}t}}}}}\quad} \\{= {{\theta(0)} + {\int{( {r + e_{r}} ){\mathbb{d}t}}}}} \\{= {{\theta(0)} + {\int{r{\mathbb{d}t}}} + {\int{e_{r}{\mathbb{d}t}}}}}\end{matrix}} & (34)\end{matrix}$

Accordingly, relation between the driving direction θ′ obtained by thecomputing unit 93 and the true driving direction θ obtained from thetrue yaw rate r is as shown in the following expression.

[Expression 35]θ′=θ+∫e_(x) dt   (35)

In the event that the error e_(r) contained in the yaw rate r′ output bythe gyro sensor 91 is white, the second term of the right-hand side ofExpression (35) is 0 in the long term, as shown in the followingexpression, so there is no particular problem. In the short term, thesecond term of the right-hand side of Expression (35) is not 0, but thiscase can be dealt with, with the processing unit 101 of the optimizingdevice 94 shown in FIG. 17.

[Expression 36]∫e_(x)dt=0   (36)

However, in the event that the error e_(r) is colored, the error e_(r)accumulates as time t passes, so the driving direction θ′ obtained fromthe computing unit 93 greatly deviates from the true driving directionθ.

That is, considering auto driving directly proceeding in a constantdirection in order to simplify description, control signals which makethe driving direction θ′ obtained from the computing unit 93 to beconstant, as indicated by the dotted line in FIG. 21, are generated atthe auto driving control unit 95 (FIG. 16).

However, in the event that the error e_(r) contained in the drivingdirection θ′ obtained from the computing unit 93 is colored, the errore_(r) accumulates as time t passes, so that the time that the drivingdirection θ′ obtained from the computing unit 93 traces a course curvingas indicated by the solid line in FIG. 21, for example, the automobileis proceeding straight ahead.

Accordingly, with the processing unit 101 of the optimizing device 94 inFIG. 20, correction parameters learning processing is performed forlearning correction parameters a₀, a₁, . . . , a_(N), for correcting thedriving direction θ′ based on learning operating signals from the user,so that the driving direction θ′ obtained from the computing unit 93supplied as input signals traces the course indicated by the solid linein FIG. 21, and correction processing is performed for correcting thedriving direction θ′ obtained from the computing unit 93 using thecorrection parameters a₀ through a_(N).

Accordingly, the correction processing and correction parameter learningprocessing which the processing unit 101 of the optimizing device 94shown in FIG. 20 performs will be described with reference to FIG. 22and FIG. 23. While the embodiment in FIG. 18 involves auto drivingproceeding straight in a constant direction, the processing unit 101 ofthe optimizing device 94 shown in FIG. 20 can be applied to auto drivingfollowing an arbitrary course.

First, the correction processing which the processing unit 101 of theoptimizing device 94 shown in FIG. 20 performs will be described withreference to the flow chart in FIG. 22.

With the correction processing, in step S151, the correction amountcalculating unit 141 calculates the correction amount using thecorrection parameters a₀ through a_(N) stored in the correctionparameter memory 146.

That is, here, the correction amount is calculated with the true drivingdirection θ being expressed as shown in Expression (37) using thecorrection parameters a₀ through a_(N) and the driving direction θ′obtained from the computing unit 93 as input signals, for example.

[Expression 37]θ=θ′+a ₀ +a ₁ t ¹ +a ₂ t ² + . . . +a _(N) t ^(N)   (37)

Accordingly, a0+a₁t¹+a₂t²+ . . . +a_(N)t^(N) is calculated at thecorrection amount calculating unit 141 as the correction amount, fromthe Expression (37). This correction amount is supplied to the computingunit 142.

In step S152, the driving direction θ′ obtained from the computing unit53 and the correction amount are added at the computing device 142, andthe added value thereof (θ in Expression (37)) is output as outputsignals, the flow waits for a sample of the next input signals to besupplied and returns to step S151, and subsequently, the same processingis repeated.

Next, the correction parameter learning processing which the processingunit 101 of the optimizing device 94 in FIG. 20 performs will bedescribed with reference to the flowchart in FIG. 23.

With the correction parameter learning processing, first, in step S161,the operating signal processing unit 110 judges whether or not learningoperating signals had been received from the operating unit 98 (FIG.16), and in the event that judgment is made that there has been noreception thereof, the flow returns to step S161.

Also, in the event that judgment is made in step S161 that learningoperating signals have been received from the operating unit 98, i.e.,in the event of that, for example, the operating unit 98 starts to beoperated, following which, without elapsing of a first time t1 orlonger, operating is performed continuously for a second time t2 orlonger, following which the operations thereof are stopped continuouslyfor a third time t3 or longer, or cases wherein following starting ofoperations of the operating unit 98, operations thereof are stoppedcontinuously for a third time t3 or longer, such that judgment can bemade that the user has operated the operating unit 98 so as to directthe automobile in the desired direction, the flow proceeds to step S162,where the tutor data generating unit 143 generates tutor data, and thestudent data generating unit 144 generates student data.

That is to say, in the event of receiving a learning operating signal,the operating signal processing unit 110 supplies a learning message tothat effect to the tutor data generating unit 143 and student datagenerating unit 144. Upon receiving the learning message, the tutor datagenerating unit 143 obtains the driving direction as input signalssupplied immediately thereafter as tutor data, and supplies this to thelearning data memory 53.

That is to say, in this case, for the tutor data, there is the need touse the driving direction following the user operating the operatingunit 98 which is the steering wheel so that the automobile heads in thedesired direction. Accordingly, the tutor data generating unit 143supplies the driving direction θ as input signals supplied the followingreception of the learning message, as tutor data, to the learning datamemory 53.

Also, the student data generating unit 52, upon receiving the learningmessage, supplies the driving direction as input signals suppliedimmediately prior to this, i.e., the driving direction immediatelybefore the automobile heads in the desired direction, as student data,to the learning data memory 53.

Following this, the flow proceeds to step S163, where the learning datamemory 53 stores the set of tutor data from the tutor data generatingunit 51 and the student data, and the flow proceeds to step S164.

In step S164, the same least-square adding as that described withExpressions (22) through (30) is performed by the correction parametercalculating unit 145, with regard to the tutor data and student data.

Note that the adding-in in step S164 is performed using the previousadding results stored in the learning information memory 55 as learninginformation, in the same manner as that described above. Also, here,adding is performed for obtaining, as θ′ in Expression (37), thecorrection parameters a₀ through a_(N) which minimize the sum of thesquare-error of the prediction value of the tutor data as θ inExpression (37) calculated using student data and the correspondingtutor data.

Following the adding in step S164, the correction parameter calculatingunit 145 stores the adding-in results as learning information, in theform of overwriting of the learning information memory 55, and the flowproceeds to step S165.

In step S165, judgment is made regarding whether or not the correctionparameter calculating unit 145 can obtain the correction parameters a₁through a_(N) from the adding results serving as learning informationstored in the learning information memory 55.

In the event that judgment is made in step S165 that the correctionparameters a₁ through a_(N) cannot be obtained, the correctionparameters calculating unit 145 supplies information to that effect tothe judgment control unit 56, and the flow proceeds to step S169. Instep S169, the judgment control unit 56 supplies and stores disable datarepresenting forbidding of correction, as correction parameters, to thecorrection parameter memory 146. The flow then returns to step S161, andsubsequently, the same processing is repeated.

Accordingly, in the event that sufficient learning information forobtaining the correction parameters a₁ through a_(N) does not exist,correction of input signals is not performed at the correcting unit 21.That is, the correction amount of input signals is 0.

On the other hand, in the event that in step S165, judgment is made thatcorrection parameters can be obtained, the flow proceeds to step S166,where the correction parameter calculating unit 145 uses the learninginformation to obtain the correction parameters a₀ through a_(N),supplies these to the judgment control unit 56, and the flow proceeds tostep S167.

In step S167, the judgment control unit 56 obtains the prediction valueof the corresponding tutor data from each of the student data stored inthe learning data memory 53, following the Expression (37) defined bythe parameter control data a₁ through a_(N) from the correctionparameters calculating unit 145, and obtains the square sum of theprediction error of that prediction value (the error as to the tutordata stored in the learning data memory 53). Further, the judgmentcontrol unit 56 obtains a normalization error wherein the square sum ofthe prediction error thereof is divided by the number of learning pairsstored in the learning data memory 53, for example, and the flowproceeds to step S168.

In step S168, the judgment control unit 56 judges whether or not thenormalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that the judgment is madein step S168 that the normalization error is greater than thepredetermined threshold value S1, i.e., in the event that the linearexpression of the Expression (37) defined by the correction parametersa₁ through a_(N) does not precisely approximate the relation betweenstudent data and tutor data stored in the learning data memory 53, theflow proceeds to step S169, where the judgment control unit 56 suppliesand stores disable data to the correction parameter memory 146 ascorrection parameters, as described above. The flow then returns to stepS161, and subsequently, the same processing is repeated.

Accordingly, even in the event that the correction parameters a₁ througha_(N) can be obtained, in the event that the Expression (37) defined bythe correction parameters a₁ through a_(N) does not preciselyapproximate the relation between the student data and tutor data storedin the learning data memory 53, the correction amount of the inputsignals x(t) is 0, in the same way as cases wherein sufficient learninginformation for obtaining the correction parameters a₁ through a_(N)does not exist.

On the other hand, in the event that judgment is made in step S168 thatthe normalization error is not greater than the predetermined thresholdvalue S1, i.e., in the event that the linear expression of theExpression (37) defined by the correction parameters a₁ through a_(N)precisely approximates the relation between student data and tutor datastored in the learning data memory 53, the flow proceeds to step S170,where the judgment control unit 56 obtains the error ε between a planeof the Expression (37) defined by the correction parameters a₁ througha_(N) from the correction parameter calculating unit 145, and the pointstipulated by the newest tutor data and student data stored in thelearning data memory 53.

The flow then proceeds to step S171, where the judgment control unit 56judges whether or not the magnitude of the error ε is greater than(equal to or more than) a predetermined threshold value S2, and in theevent that judgment is made that this is not greater, step S172 isskipped, the flow proceeds to step S173, where the judgment control unit56 outputs the correction parameters a₀ through a_(N) obtained in stepS166 to the correction parameters memory 146. In this case, thecorrection parameters memory 146 stores the correction parameters a₁through a_(N) from the judgment control unit 56 in the form ofoverwriting, and the flow returns to step S161.

On the other hand, in the event that judgment is made in step S171 thatthe magnitude of the error ε is greater than the predetermined thresholdvalue S2, the flow proceeds to step S172, where the judgment controlunit 56 controls the correction parameter calculating unit 145 tore-calculate the correction parameters a₀ through a_(N), using only thenewest tutor data and student data stored in the learning data memory53. The flow then proceeds to step S173, where the judgment control unit56 outputs the correction parameters a₀ through a_(N) obtained in stepS172 to the parameter control unit data memory 37, to store in the formof overwriting, and the flow returns to step S161.

That is, in the same way as the embodiment in FIG. 8, with theembodiment in FIG. 23 as well, in step S170, the error ε is obtainedbetween a plane defined by the Expression (37) from the correctionparameters a₀ through a_(N) obtained from the tutor data and studentdata provided so far, and the point stipulated by the newest tutor dataand student data.

In the event that the magnitude of the error E is not greater than thethreshold value S2, the plane of the Expression (37) defined by thecorrection parameters a₀ through a_(N) obtained in step S166 isconsidered to relatively precisely approximate the points stipulated bytutor data and student data provided so far, including the pointstipulated by the newest tutor data and student data, so the correctionparameters a₀ through a_(N) are stored in the correction parametermemory 146.

On the other hand, in the event that the magnitude of the error ε isgreater than the threshold value S2, the point stipulated by the newesttutor data and student data is considered to relatively greatly deviatefrom the plane of the Expression (37) defined by the correctionparameters a₀ through a_(N) obtained in step S166, so in step S172, thejudgment control unit 56 re-calculates the correction parameters a₀through a_(N) using only the newest tutor data and student data storedin the learning data memory 53.

Accordingly, in this case as well, learning is performed for thecorrection parameters a₀ through a_(N) of the Expression (37) based onlearning operating signals supplied according to operations made by theuser, so operations of the user can be learned without the user knowing,and further, processing which is optimal for the user can be performedusing the learning results.

Further, in this case, in the event that the error contained in therunning direction output by the computing unit 93 (FIG. 16) is colored,the automobile can be made to perform auto-driving following thepredetermined set course.

Also, with the processing unit 101 of the optimizing device 94 in FIG.20, the correction parameters used in the correction processing (FIG. 6)performed at the correcting unit 21 are changed so that the actualdriving direction of the automobile follows the set course, followingthe operations of the user made at the operating unit 98. That is, uponthe user operating the operating unit 98 so that the driving directionof the automobile heads in the desired direction, learning is performedof correction parameters, with each of the driving directions θimmediately before and immediately after operating of the operating unit98 output by the computing unit 43 (FIG. 16) as input signals, asstudent data and tutor data, respectively, thereby changing thecorrection parameters. The changed correction parameters are supplied tothe correcting unit 21, the correction amount is calculated at thecorrecting unit 21 using the correction parameters, and correctionprocessing (FIG. 22) is performed for the input signals depending on thecorrection amount. Accordingly, in the event that the user operates theoperating unit 98, the correction parameters of the Expression (37) arechanged by the operations of the user, so that the contents of theprocessing (correction processing) represented by Expression (37) alsochange as a matter of course, so it can be said that the “contents ofprocessing” are being changed so that the driving direction desired bythe user can be obtained, following operations of the user, with theprocessing unit 101 of the optimizing device 94 shown in FIG. 20, aswell.

Further, with the optimizing device 94 in FIG. 20, in the event thatsufficient learning pairs are not input from the user, or in the eventthat learning pairs capable of highly precise approximation are notinput, the correction amount of input signals at the correcting unit 21is made to be 0, and in the event that learning pairs capable of highlyprecise approximation are input from the user, correction of the inputsignals is performed by a correction amount obtained from correctionparameters obtained by learning being performed using the learningpairs. That is, the algorithms for calculating weighting used forcorrection processing by the correcting unit 21 change between caseswherein sufficient numbers of learning pairs or learning pairs capableof precise approximation are not obtained, and cases wherein learningpairs capable of highly precise approximation are obtained.

Accordingly, with the processing unit 101 of the optimizing device 94 inFIG. 20, as well, the “contents of processing”, and further, the“structure of processing”, is also changed following user operations,whereby the automobile performs auto-driving in the driving directionfollowing the set course.

Now, with the embodiment in FIG. 23 (as with the embodiments in FIG. 8and FIG. 13), in step S170, the error E between the plane of theExpression (37) defined by the correction parameters a₀ through a_(N)from the correction parameters calculating unit 145, and the pointstipulated by the newest tutor data and student data, is obtained, andsubsequent processing is performed, but an arrangement may be madewherein, in step S170, the error ε between the plane of the Expression(37) defined by the correction parameters a₀ through a_(N) obtained instep S166 before multiple sets of recent tutor data and student data aresupplied, and each of points stipulated by multiple sets of recent tutordata and student data, is obtained, and subsequent processing isperformed based on the multiple errors ε.

Note that the processing unit 101 of the optimizing device 94 in FIG. 16may be configured to use the processing unit 11 of the optimizing device1 shown in FIG. 10, for example, besides those shown in FIG. 17 and FIG.20.

Next, FIG. 24 illustrates a configuration example of another embodimentof an auto-driving device to which he optimizing device shown in FIG. 1has been applied. Note that in the figure, the parts corresponding tothose in FIG. 16 are denoted by the same reference numerals, anddescription thereof will be omitted where appropriate in the following.That is to say, the auto-driving device shown in FIG. 24 is configuredin the same way as that shown in FIG. 16, other than having an internalinformation generating unit 161 newly provided to in the optimizingdevice 94, and further having a display device 171 newly provided.

In the same way as the interval information generating unit 171 in FIG.14, the internal information generating unit 161 reads out the internalinformation from the processing unit 1, converts this into imageinformation, and outputs to the display unit 171. The display unit 171displays the interval information supplied from the internal informationgenerating unit 161 in a predetermined display format.

In the embodiment shown in FIG. 24, the processing unit 101 can beconfigured as shown in FIG. 17 are FIG. 20. In the event that theprocessing unit 101 shown in FIG. 24 is configured in the same way asthat shown in FIG. 17, the same processing as that in FIG. 17 isperformed except for the correction parameter computing processing. Now,the correction parameter computing processing which the processing unit101 shown in FIG. 24 performs when configured as shown in FIG. 17, willbe described with reference to the flowchart in FIG. 25.

With steps S191 through S197, the same processing as that in each ofsteps Sill through S117 in FIG. 18 is performed.

Following the processing in step S197, the flow proceeds to step S198,and interval information is displayed to the display unit 171. That is,more specifically, for example, the internal information generating unit161 reads out the weight W stored in the weighting memory 31 (FIG. 17)as internal information, converts this into image signals which can bedisplayed on the display unit 171, and outputs and displays (presents)on the display unit 171. Following the processing step S198, the flowreturns to step Sill, and subsequently, the same processing is repeated.

Due to the processing described with the flowchart in FIG. 25, theweight W which is internal information regarding the processing of theprocessing unit 101 is displayed (presented) on the display unit 171,and consequently, the user can operate the operating unit 98 whilewatching the display, so that optimal auto-driving is performed.

Also, in the above-described case, the weight W is displayed, but otherinternal information may be displayed (presented) on the display device171 with this internal information generating unit 161, for example,parameter control data a and b can be read out from the parametercontrol data memory 37 and displayed. Also, an arrangement may be madewherein the weight selected by the selecting unit 41 is weight obtainedfrom the parameter control data a and b obtained by learning beingperformed using learning pairs, or weight obtained from the inputreliability or output reliability.

Next, in the event that the processing unit 101 of FIG. 24 is configuredas shown in FIG. 20, processing the same as that performed in FIG. 20 isperformed except for the correction parameter learning processing. Now,correction parameter learning processing in the case that the processingunit 101 shown in FIG. 24 is configured as shown in FIG. 20, will bedescribed, with reference to the flowchart shown in FIG. 26.

In steps S211 to S223, processing the same as that in each of steps S161through S172 in FIG. 23 is performed.

Following the processing in step S219 and step S223, the flow proceedsto step S224, where, for example, the internal information generatingunit 161 reads out the correction parameters a₀ through a_(N) stored inthe correction parameter memory as internal information, converts thisinto image signals that can be displayed on the display unit 171, anddisplays on the display unit 171. At this time, the correctionparameters a₀ through a_(N) configured of multiple parameters, anotherarrangement may be made wherein a display is-made in the form of a bargraph, for example, with parameters as the horizontal axis and thevalues as the vertical axis, as shown in FIG. 27. Also, as shown in FIG.28, for example, the correction parameters a₀ through a_(N) may bedisplayed with two arbitrary correction parameters a_(i) and a_(j) asthe horizontal axis and vertical axis, respectively. Further, thearrangement may be made wherein the correction parameters to use as thehorizontal axis and vertical axis may be selected by the user.

Subsequently, the flow returns to step S211, and subsequently, the sameprocessing is repeated.

As described above, due to the correction parameter processing describedwith reference to the flowchart in FIG. 26, the correction parameters a₀through a_(N) are displayed, whereby the correction parameters a₀through a_(N) are displayed as internal information with the processingunit 101 of the optimizing device 94 shown in FIG. 24, so consequently,the user can operate the operating unit 98 while watching the display,so that optimal auto-driving is performed.

Further, with the internal information generating device 161, internalinformation other than the correction parameters a₀ through a_(N) may bedisplayed.

Also, in the embodiment shown in FIG. 26, in the event that the flowproceeds to step S224 following the processing in step S219, thecorrection parameters a₀ through a_(N) serving as the internalinformation are displayed as being 0.

Next, an optimizing device 201 as another embodiment of the optimizingdevice shown in FIG. 1 will be described with reference to FIG. 29. Theoptimizing device 201 is configured of a processing unit 211, and is forremoving noise from image signals and the like, for example serving asinput signals, and optimizing displayed image signals. Now, in thisexample, image signals will be described as an example of as primaryinput signals, the signals are not restricted to image signals, and maybe other signals as well.

The processing unit 211 is configured of a learning unit 221 and amapping processing unit 222. Operating signals are supplied to thelearning unit 221 of the processing unit 211 from the operating unit202, and the learning unit 221 learns coefficient sets necessary for theprocessing with the mapping processing unit 222 based on the operatingsignals, and stores these in the coefficient memory 235. As for the normof learning of learning unit 211 (learning norm), for example, theleast-N′th-power error method (least-N′th-power method) may be used. Theleast-N′th-power error method will be described later.

The mapping processing unit 222 performs mapping processing for mapping(converting) the input signals to predetermined output signals. That is,the mapping processing unit 222 takes, of the image signals serving asoutput signals, a pixel yet to be obtained as a pixel of interest,extracts that tap (at least one or more pixel necessary for processing,also referred to as a sample) corresponding to the pixel of interestfrom the image signals which are input signals, and executes the sum ofproducts computing processing of coefficient sets stored in thecoefficient memory 235, thereby obtaining the pixel of interest. Themapping processing unit 222 performs processing (mapping) the same aswith pixels making up the image signals serving as output signals,generates image signals serving as output signals thereby which areoutput to the display unit 203, and displayed.

The operating unit 202 is operated by the user, and supplies operatingsignals corresponding to the operations, to the learning unit 221. Thedisplay unit 203 displays pixel signals as output signals output by themapping processing unit 202.

Next, a detailed configuration of the learning unit 221 shown in FIG. 29will be described, with reference to FIG. 30. The tutor data generatingunit 231 generates tutor data serving as a tutor for learning, frominput signals, and outputs this to a least-N′th-power error coefficientcomputing unit 234. The student data generating unit 232 generatesstudent data serving as a student for learning from the input signals,and outputs this to a prediction tap extracting unit 233. This tutordata and student data is generated, for example, by the tutor data 231performing no processing with the input signals, or by the student datagenerating unit 232 causing deterioration of input data by apredetermined thinning-out processing or LPF (Low Pass Filter) or thelike, etc., and is not restricted to the above-described configurationas long as student data is generated as that which is deteriorated withregard to the tutor data. Accordingly, besides that described above, forexample, in the event that the tutor data generating unit 231 subjectsthe input signals to predetermined thinning-out or LPF processing, it issufficient for the student data generating unit 232 to performthinning-out or LPF processing to a degree greater than that performedby the tutor data generating unit 231. Also, an arrangement may be madewherein the input signals are used without change as tutor data, andinput signals with noise superimposed thereupon is used as student data.

The prediction tap extracting unit 233 then sequentially takes pixelsconfiguring the image signals serving as tutor data as pixels ofinterest, extracts at least one or more pixels (taps) in a predeterminedpositional relation with pixel of interest as a prediction tap fromimage signals serving as student data, and outputs this to theleast-N′th-power error coefficient computing unit 234.

The least-N′th-power error coefficient computing unit 234 computescoefficient sets from the prediction tap and tutor data byleast-N′th-power error, based on the operating signals representinginformation specifying the value of the exponent N necessary for theleast-N′th-power error coefficient computing processing input from theoperating unit 202, outputs this to the coefficient memory 235, andstores (performs overwriting storing as suitable).

The coefficient memory 235 stores the coefficient sets supplied from theleast-N′th-power error coefficient computing unit 234, and outputs thisto the mapping processing unit 222 where appropriate.

Next, the configuration of the mapping processing unit 222 in FIG. 29will be described with reference to FIG. 31. The tap extracting unit 251of the mapping processing unit 222 configures a prediction tap with thesame tap configuration as that configured by the prediction tapextracting unit 233 shown in FIG. 30 by sequentially extracting pixelsconfiguring image signals serving as output signals as pixels ofinterest from image signals serving as input signals, as prediction tapsof (the pixel value of) pixels in a predetermined positional relationwith pixels of interest, and outputs this to the sum of productscomputing unit 252. The sum of products computing unit 252 generatespixels of interest by performing the sum of products computation betweenthe values of extracted prediction taps (pixels) input from the tapextracting unit 251, and coefficient sets stored in the coefficientmemory 235 of the learning unit 221, and outputs this to the displayunit 203 (FIG. 29).

Now, coefficient computation by least-N′th-power error performed by theleast-N′th-power error coefficient computing unit 234 shown in FIG. 30will be described. A case wherein the exponent N=2 in theleast-N′th-power error method is commonly referred to as theleast-square error method (least-square-power error method). That is tosay, in a case wherein tutor data serving as a pixel of interest isrepresented by y, an M number of pieces of student data configuring aprediction tap by xi (i=1, 2, . . . , M), and a predetermined M numberof coefficients as w_(i), respectively, and the prediction value y′ ofthe tutor data y is a linear combination (sum of products computation)w₁x₁+w₂x₂+ . . . +w_(M)x_(M) of a prediction tap x_(i) and apredetermined coefficient w_(i), a coefficient set w₁, w₂, . . . , w_(M)which minimizes, as shown in FIG. 32, the sum of the square of the errorbetween the tutor data y indicated by black dots and the predictionvalues y′ indicated by white dots (in the absolute value of differencebetween the true value y serving as tutor data, and the prediction errory′ thereof, indicated by arrows in the figure), is obtained.

In the event of changing the value of the exponent N in least-N′th-powererror, the error of a prediction value y′ with great error greatlyaffects the sum of the N′th-power error in the event that the exponent Nis great, for example, so consequently, with the least-N′th-power errormethod, coefficients in the direction of whereby a such a predictionvalue y′ with a great error is saved (coefficients whereby the error ofthe prediction value y′ with a great error is reduced) is obtained.However, the error of a prediction value y′ with little error has littleeffect on the sum of the N′th-power, and is not given muchconsideration, and consequently tends to be ignored. Conversely, in theevent that the exponent N is small, the error of a prediction value y′with great error affects the sum of the N′th-power less than caseswherein the exponent N is great, and the error of a prediction value y′with little error affects the sum of the N′th-power even less.Consequently, according to the least-N′th-power error method,coefficients in the direction of reducing the error of the predictionvalue with small error is obtained in comparison with cases wherein theexponent N is great.

Note that the change in effect which the error of the prediction valuey′ has on the sum of the N′th-power error is due to the error of theprediction value y′ to the N′th power, which may hereafter be referredto as the nature of N′th-power error.

As described above, there is a qualitative tendency of coefficient setsobtained with least-N′th-power from the exponent N, so coefficient setsfor executing mapping processing suiting the preferences of the user canbe obtained by changing the exponent N to obtain coefficient sets withthe least-N′th-power method (coefficient sets which makes the outputsignals mapped with regard to the input signals, to be those preferredby the user). However, in reality, it is extremely difficult to computecoefficient sets which minimize the sum of the N′th-power error of aprediction value y′, with the method other than least-N′th-power error,i.e., a method wherein the exponent is other than N=2.

Now, the reason why computing coefficient sets which minimize the sum ofthe N′th-power error of the prediction value y′ with methods other thanleast-square-power error will be described.

The sum of the error of the prediction value y′ to the N′th-power (thesum of the N′th-power error) can be represented by Expression (38).$\begin{matrix}{\lbrack {{Expression}\quad 38} \rbrack\quad{E = {\sum\limits_{sample}^{\quad}{\mathbb{e}}^{N}}}} & (38)\end{matrix}$

Here, E indicates the sum of the number of samples of the N′th power ofthe error e between the true value y serving as tutor data and theprediction value y′.

On the other hand, with the present embodiment, as described above, theprediction value y′ of the true value y is defined by a linearcombination of the prediction tap x_(i) and a predetermined coefficientwi, i.e., as with the following Expression (39). $\begin{matrix}{\lbrack {{Expression}\quad 39} \rbrack\quad{Y^{\prime} = {{{w_{1}x_{1}} + {w_{2}x_{2}} + \ldots + {w_{M}x_{M}}} = {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}}}} & (39)\end{matrix}$

Now, the coefficients w₁, w₂, w₃, . . . , w_(M), may hereafter bereferred to as prediction coefficients. A set of the predictioncoefficients is the coefficients set stored in the coefficient memory235 shown in FIG. 30. Also, for the prediction taps x₁, x₂, x₃, . . . ,x_(M), pixels close space-wise or time-wise to the position of the imageserving as student data corresponding to the pixel (true value) y of theimage serving as tutor data, can be employed.

In this case, the error e in Expression (38) can be represented by thefollowing Expression (40). $\begin{matrix}{\lbrack {{Expression}\quad 40} \rbrack\quad{e = {{{y - y^{\prime}}} = {{y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}}}}}} & (40)\end{matrix}$

With the least-N′th-power error method, there is the need to obtain theprediction coefficients w₁, w₂/w₃, . . . , w_(M) which minimize the sumE of the N′th-power error indicated in the following Expression (41) andExpression (42) yielded from Expression (40). Note that Expression (41)is an expression indicating the sum E in the event that the exponent Nis an odd number, and Expression (42) is an expression indicating thesum E in the event that the exponent N is an even number.$\begin{matrix}{\lbrack {{Expression}\quad 41} \rbrack\quad{E = {{\sum\limits_{sample}^{\quad}{\mathbb{e}}^{N}} = {\sum\limits_{sameple}^{\quad}{{y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}}}^{N}}}}} & (41) \\{\lbrack {{Expression}\quad 42} \rbrack\quad{E = {{\sum\limits_{sample}^{\quad}{\mathbb{e}}^{N}} = {{\sum\limits_{sample}^{\quad}{{y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}}}^{N}} = {\sum\limits_{sample}^{\quad}( {y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}} )^{N}}}}}} & (42)\end{matrix}$

Now, in the case of Expression (41), that is, in the event that theexponent N is an odd number, the sum E is the same value regardless ofthe notation of the difference y−y′ if the sum of the magnitude of thedifference y−y′ between the true value y and the prediction value y′ isthe same value, and consequently, the prediction coefficients w₁, w₂,w₃, . . . , w_(M) which minimize the sum E cannot be obtained. That isto say, the sum E is a function containing absolute values, and is, forexample a function such as shown in FIG. 33. Accordingly, the predictioncoefficients w₁, w₂, w₃, . . . , w_(M) which minimize the sum E cannotbe obtained except by a full search. FIG. 33 represents the change of aprediction coefficient w_(i) and the sum E in the event that theexponent N=1.

On the other hand, in the case of Expression (42), i.e., in the eventthat the exponent N is an even number, the sum E always satisfies E≧0,so as indicated in the following Expression (43), the minimal value canbe obtained by setting an expression wherein the sum E in Expression(42) subjected to partial differentiation by the prediction coefficientsw_(i) is 0. $\begin{matrix}{\lbrack {{Expression}\quad 43} \rbrack\quad\begin{matrix}{\frac{\partial E}{\partial w_{i}} = {\sum\limits_{sample}^{\quad}{{N( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} )}^{N - 1} \cdot ( {- x_{i}} )}}} \\{= {{- N}{\sum\limits_{sample}^{\quad}{x_{i}( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} )}^{N - 1}}}} \\{= 0}\end{matrix}} & (43)\end{matrix}$

Accordingly, from Expression (43), the solution, i.e., the predictioncoefficients w₁, w₂, w₃, . . . , w_(M) which minimize the sum E of theN′th-power error, can be obtained by solving the expression indicated inthe following Expression (44). $\begin{matrix}{\lbrack {{Expression}\quad 44} \rbrack\quad{{{\sum\limits_{sample}^{\quad}{x_{i}( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} )}^{N - 1}} = 0},( {{i = 1},2,\ldots\quad,M} )}} & (44)\end{matrix}$

With regard to this Expression (44), in the event that the exponent N=2for example, i.e., in the event of obtaining the solution with theso-called least-square error method, substituting 2 for the exponent Nin Expression (44) and solving the following Expression (45) issufficient. $\begin{matrix}{\lbrack {{Expression}\quad 45} \rbrack\quad{{{\sum\limits_{j = 1}^{M}{\sum\limits_{sample}^{\quad}{x_{i}x_{j}w_{j}}}} = {\sum\limits_{sample}^{\quad}{x_{i}y}}},( {{i = 1},2,\ldots\quad,M} )}} & (45)\end{matrix}$

As shown in the following. Expression (46), the expression in Expression(45) can be represented in the form of a matrix expression, and iscalled a normal equation. Also, in the event that the exponent N=2, theminimal value of the sum E is uniquely determined, and the smallestvalue thereof is the minimal value of the sum E. If a system of linearequations of a number equal to the number of the prediction coefficientsw₁, w₂, w₃, . . . , w_(M) (M in this case) is configured from the normalexpression in Expression (45), the system of linear equations can besolved by the Cholesky method or the like, for example, and theprediction coefficients w₁, w₂, w₃, . . . , w_(M) can be obtained.$\begin{matrix}{\lbrack {{Expression}\quad 46} \rbrack\quad{{\begin{pmatrix}{\sum\limits_{sample}^{\quad}{x_{1}x_{1}}} & {\sum\limits_{sample}^{\quad}{x_{1}x_{2}}} & \cdots & {\sum\limits_{sample}^{\quad}{x_{1}x_{M}}} \\{\sum\limits_{sample}^{\quad}{x_{2}x_{1}}} & {\sum\limits_{sample}^{\quad}{x_{2}x_{2}}} & \cdots & {\sum\limits_{sample}^{\quad}{x_{2}x_{M}}} \\\vdots & \vdots & \vdots & \vdots \\{\sum\limits_{sample}^{\quad}{x_{M}x_{1}}} & {\sum\limits_{sample}^{\quad}{x_{M}x_{2}}} & \cdots & {\sum\limits_{sample}^{\quad}{x_{M}x_{M}}}\end{pmatrix}\begin{pmatrix}w_{1} \\w_{2} \\\vdots \\w_{M}\end{pmatrix}} = \begin{pmatrix}{\sum\limits_{sample}^{\quad}{yx}_{1}} \\{\sum\limits_{sample}^{\quad}{yx}_{2}} \\\vdots \\{\sum\limits_{sample}^{\quad}{yx}_{M}}\end{pmatrix}}} & (46)\end{matrix}$

Also, in order to solve the normal equation in Expression (46), it isnecessary that the matrix having the sum of the product of predictiontaps x_(i)x_(j) (Ex_(i)x_(j)) at the left-hand side of the Expression(46) as a component is regular.

In the event that the exponent N is an even number and also is 4 ormore, the Expression (42) can be expressed as shown in the followingExpression (47).$\lbrack {{Expression}\quad 47} \rbrack\quad\begin{matrix}{{{\sum\limits_{r = 0}^{N - 1}{\sum\limits_{example}{{x_{i} \cdot ( {- 1} )^{r}}{{{}_{N - 1}^{}{}_{}^{}} \cdot y^{N - 1 - r} \cdot ( {\sum\limits_{j = 1}^{N}{w_{j}x_{j}}} )^{r}}}}} = 0},( {{i = 1},2,\ldots\quad,M} )} & (47)\end{matrix}$

With the expression shown in Expression (47), this forms a system ofequations of a high order, so the prediction coefficients w₁, w₂, w₃, .. . , w_(M) cannot be obtained by solving a system of linear expressionsas with the case wherein the exponent N is N=2.

As described above, in the event that the exponent N is other than N=2,the prediction coefficients w₁, w₂, w₃, . . . , w_(M) which minimize thesum of N′th-power error indicated in Expression (38) generally cannot bereadily obtained.

Accordingly, the least-N′th-power error coefficient computing unit 234of the learning unit 221 computes the prediction coefficients by thefollowing two least-N′th-power error methods. Which of the twoleast-N′th-power error methods to employ can be specified by, forexample, the user operating the operating unit 202 (FIG. 29).

First, a first method (hereafter also referred to as the directtechnique) will be described. As indicated in the following Expression(48), the sum E wherein the item of the error e² has been multiplied bythe weight α_(s) is defined as the sum of the N′th-power error insteadof the Expression (38).$\lbrack {{Expression}\quad 48} \rbrack\quad\begin{matrix}{E = {{\sum\limits_{example}e^{N}} = {\sum\limits_{sample}{\alpha_{s}e^{2}}}}} & (48)\end{matrix}$

That is, the e^(N) which is error to the N′th power is defined by theproduct of the weight α_(s) and the error squared e².

In this case, the prediction coefficients w₁, through w_(M) whichminimize the sum E of N′th-power error in Expression (48) can beobtained by making the weight α_(s) in Expression (48) to be a functionof the prediction value y′ obtained by the linear expression inExpression (39), from the prediction coefficients w₁, w₂, w₃, . . . ,w_(M) obtained in the event that the exponent N is N=2 as shown in thefollowing Expression (49), for example.

[Expression 49]α_(s) =f(y′)   (49)

While various function can be conceived for the weight α_(s), there isthe need to employ a function wherein the error to the N′th powere^(N)=α_(s)e² defined in Expression (48) satisfies the above-describednature of the N′th-power error, and the function shown in the followingExpression (50), for example, can be employed.

[Expression 50]α_(s) =ax _(s) ^(c) +b   (50)

Now, x_(s) represents a value wherein the error of the prediction valuey′ computed by Expression (39) from the prediction coefficients w₁,through w_(M) obtained by least-square error (hereafter may be referredto as error from least-square norm) is normalized to a range of 0through 1.0, and the weight α_(s) in the Expression (50) defined withthe error x_(s) of the prediction value y′ is as shown in FIG. 34.

The coefficient a is an item for controlling the effect which the errorx_(s) from the least-square norm has on the N′th-power error e^(N), andin the event that the coefficient a is a=0, the weight α_(s) is ahorizontal line with an inclination of 0 in FIG. 34. In this case, theeffect which the error x_(s) from the least-square norm has on theN′th-power error e^(N) is constant regardless of how large or how smallthe error x_(s) from the least-square norm is, and the predictioncoefficient w_(i) which minimizes the sum E in Expression (48) logicallyis the same as that obtained by least-square error. Accordingly, makinga=0 essentially realizes the least-square error method. Also, in theevent that coefficient a >0 holds,

-   -   the effect which the error x_(s) from the least-square norm has        on the N′th-power error e^(N) in Expression (48) is greater the        greater the error x_(s) is, and is smaller the smaller the error        x_(s) is. On the other hand, in the event that coefficient a<0        holds, the effect which the error x_(s) from the least-square        norm has on the N′th-power error e^(N) in Expression (48) is        smaller the greater the error x_(s) is, and is greater the        smaller the error x_(s) is.

That is to say, in the event that the coefficient a is positive, theN′th-power error e^(N) in Expression (48) has the same nature as makingthe exponent N to be large, and in the event that the coefficient a isnegative, this has the same nature as making the exponent N to be small.Accordingly, the N′th-power error e^(N) in Expression (48) has the samenature as the N′th-power error e^(N) in Expression (38), so predictioncoefficients which minimize the sum E of the N′th power error inExpression (48) essentially is the prediction coefficients whichminimize the sum E of the N′th power error in Expression (38).

Also, in the event that the coefficient a is 0, the least-square erroris realized, as described above. That is, the exponent N is 2, but withcases wherein the coefficient a is positive, this makes for caseswherein the exponent N is N>2, and with cases wherein the coefficient ais negative, this makes for cases wherein the exponent N is N<2. Thiscoefficient a greatly affects the exponent N in least-N′th-power error,as with the later-described coefficient c.

The coefficient b is a correction item, and the overall function value(weight α_(s)) in FIG. 34 changes in the vertical direction according tothe value of the coefficient b. Also, the coefficient b does not havethat much of an effect on the exponent N with least-N′th-power error.

The coefficient c is an item which converts the scaling of the axes,i.e., an item which changes the way the weight α_(s) is appropriated tothe error x_(s) from the least-square norm, wherein the greater thevalue of the coefficient c is the more precipitous the change in theweight α_(s) is, and the smaller the value of the coefficient c is thegentler the change in the weight α_(s) is. Accordingly, the effect whichthe error x_(s) from the least-square norm has on the N′th-power errore^(N) in Expression (48) due to the coefficient c changing is the sameas that of the coefficient a changing, so the N′th-power error e^(N) inExpression (48) can be provided with the same nature as the N′th-powererror e^(N) in Expression (38) with the coefficient c, as well. That isto say, the exponent N of the least-N′th-power error can be affected bythe coefficient c.

Note that the line in FIG. 34 indicates a case wherein c=1 and a>0 (b isarbitrary), and the curve in FIG. 34 indicates a case wherein c≠1 anda>0 (b is arbitrary).

The coefficients a, b, and c, which stipulate the weight as inExpression (50) may be changed by the user operating (setting) theoperating unit 202, and the weight α_(s) in Expression (50) changes bychanging the coefficients a, b, and c. This change in the weight α_(s)causes the α_(s)e² in Expression (48) to essentially (equivalently)function as the error to the N′th power e^(N) with regard to apredetermined exponent N, and consequently, the prediction coefficientswhich minimize the sum E of N′th-power error in Expression (50), i.e.,the prediction coefficient w_(i) of the norm by least-N′th-power error,to be obtained.

Also, with the above-described first method, variously changing thecoefficients a, b, and c, which determine the weight α_(s) essentiallychanges the exponent N so as to obtain the prediction coefficients byleast-N′th-power error, so the exponent N is not restricted to integers,and the prediction coefficients can be obtained in the event that theexponent N is another real number such as a decimal, e.g., wherein theexponent N=2.2.

Next, a second method for computing prediction coefficients byleast-N′th-power error (hereafter also referred to as the recursivemethod) will be described. With the first method, the squared error e²multiplied by the weight α_(s) is employed as the N′th-power error asshown in Expression (48), but with the second method, a solutionobtained with lower order least-N′th-power error is used, and a solutionby higher order least-N′th-power error can be obtained with a recursivetechnique.

That is, the prediction coefficient wi which minimizes the sum E ofsquare-error in the following Expression (51) can be obtained with theleast-square error method as described above, with the prediction valuey′ computed by the Expression (39) using the prediction coefficientw_(i) obtained by this least-square error being represented by y₁(hereafter may be referred to as prediction value from least-squarenorm).

[Expression 51]E=Σe²   (51)

Next, let us consider a sum E of a cubic error expressed in thefollowing Expression (52), for example.

[Expression 52]E=Σ|e| ³=Σ(|y−y ₁|)e ²   (52)

Obtaining the prediction coefficient w_(i) which minimizes the sum E ofthe cubic-error in Expression (52) means obtaining a solution byleast-cubic error, wherein the cubic-error |e³| is expressed in the formof the product of the square-error e² and the error |y−y₁| between theprediction value y₁ from the least-square norm and the true value y, asshown in Expression (52). The |y−y₁| in Expression (52) can be obtainedas a constant, so the prediction coefficient w_(i) which minimizes thesum E of the cubic-error in Expression (52) can actually be obtained byleast-square error.

In the same way, let us consider a sum E of a biquadrate error expressedin Expression (53).

[Expression 53]E=Σe ⁴=Σ(|y−y ₂|)² e ²   (53)

Obtaining the prediction coefficient w_(i) which minimizes the sum E ofthe biquadrate-error in Expression (53) means obtaining a solution byleast-biquadrate error, wherein, with the prediction value y′ computedby Expression (39) using the prediction coefficient w_(i) whichminimizes the sum E of the cubic-error in Expression (529) beingrepresented by y₂ (hereafter may be referred to as prediction value fromleast-cubic norm), the biquadrate-error e⁴ can be expressed in the formof the product of the square-error e² and the squared error |y−y₂|²between the prediction value y₂ from the least-cubic norm and the truevalue y (hereafter may be referred to as square-error from least-cubicnorm), as shown in Expression (53). The squared error |y−y₂|² from theleast-cubic norm in Expression (53) can be obtained as a constant, sothe prediction coefficient w_(i) which minimizes the sum E of thebiquadrate-error in Expression (53) can actually be obtained byleast-square error.

This is also true for the following Expression (54).

[Expression 54]E=Σ|e| ⁵=Σ(|y−y ₃|)³ e ²   (54)

Obtaining the prediction coefficient w_(i) which minimizes the sum E ofthe fifth-power-error in Expression (54) means obtaining a solution byleast-fifth-power error. With the prediction value y′ computed byExpression (39) using the prediction value w_(i) which minimizes the sumE of the biquadrate-error in Expression (53) being represented by y₃(hereafter may be referred to as prediction value from least-biquadratenorm), the fifth-power-error e⁵ can be expressed in the form of theproduct of the square-error e² and the cubed error |y−y₃|³ between theprediction value y₃ from the least-biquadrate norm and the true value y(hereafter may be referred to as cubic-error from least-biquadratenorm), as shown in Expression (54). The cubed error of theleast-biquadrate norm in Expression (54) can be obtained as a constant,so the prediction coefficient w_(i) which minimizes the sum E of thefifth-power-error in Expression (54) also can actually be obtained byleast-square error.

The solution (prediction coefficient w_(i)) can also be obtained in thesame manner with cases of least-N′th power error wherein the exponent Nis 6 or a higher order.

As described above, with the second method, a prediction value computedusing a prediction coefficient obtained with lower orderleast-N′th-power error is used to obtain a solution of higher orderleast-N′th-power error, and this is recursively repeated, therebyobtaining a solution with higher order least-N′th-power error. Note thatwith the above-described case, the solution from least-N′th-power erroris obtained using a prediction value computed using a predictioncoefficient obtained by N-1′th-power error which is lower thereby byjust the order of 1, but the solution by least-N′th-power error may beobtained using a prediction value computed using a predictioncoefficient obtained by least-N′th-power error of an arbitrary orderlower than that. That is, in the case of Expression (53), |y−y₁| may beused instead of |y−y₂|, and in the case of Expression (54), |y−y₂| or|y−y₁| may be used instead of |y−y₃|.

Also, in the second method, the N′th-power-error e^(N) can be expressedin the form of the product of the square-error e² and the N-2′th-powererror |y−y′|^(N-2), so as with the first method, a solution can beobtained by least-N′th error with an arbitrary exponent N, e.g., whereinthe exponent N=2.2.

Next, the image optimizing processing with the optimizing device 201 inFIG. 29 will be described with reference to the flowchart in FIG. 35.This image optimizing processing is made up of learning processing andmapping.

In the learning processing, in Step S230, the judgment is made regardingwhether or not the user has operated the operating unit 202, and in theevent that judgment is made that this has not been operated, the flowreturns to Step S230. Also, in Step S230, in the event that judgment ismade the operating unit 202 has been operated, the flow proceeds to StepS231.

In Step S231, the tutor data generating unit 231 of learning unit 221generates tutor data from the input signals and outputs this to theleast-N′th-power error coefficient computing unit 234, while the studentdata generating unit 232 generates student data from the input signalsand outputs this to the prediction tap extracting unit 233, and the flowproceeds to Step S232.

Now, as for data used for generating the student data and tutor data(hereafter may be referred to as learning data), input signals input toa point back in time from the current time by a predetermined amount oftime may be employed, for example. Also, for the learning data, anarrangement may be made wherein dedicated data is stored beforehand,instead of using input signals.

In Step S232, with each piece of tutor data as pixel of interest, theprediction tap extracting-unit 233 generates prediction tap from thestudent data input by the student data generating unit 232 with regardto each pixel of interest, and outputs this to the least-N′th-powererror coefficient computing unit 234, and the flow proceeds to StepS233.

In Step S233, the least-N′th-power error coefficient computing unit 234judges whether or not operating signals specifying computation of acoefficient set by least-N′th-power error with the recursive method(second method) has been input by the operating unit 202, and in theevent that, for example, the operating unit 202 has been operated by theuser and a method which is not the recursive method has been specified,i.e., the direct method (first method), the flow proceeds to Step S234,where the judgment is made regarding whether or not coefficients a, b,and c, which specify the weight α_(s) in Expression (50) (specify theexponent N), the processing is repeated until there is input thereof,and in the event that judgment is made that, for example, the operatingunit 202 has been operated by the user and the values specifying thecoefficients a, b, and c, have been input, the processing proceeds toStep S235.

In Step S235, the least-N′th-power error coefficient computing unit 234essentially solves the problem of minimizing the above-describedExpression (48) in the state of the coefficients a, b, and c with theweight α_(s) input by the least-square error method, thereby obtainingthe prediction coefficients w₁, w₂, w₃, . . . , w_(M), has a solutionwith least-N′th-power error for the exponent N corresponding to theweight α_(s), i.e., a coefficient set, and stores this in thecoefficient memory 235, and the flow returns to Step S230.

On the other hand, in Step S213, in the event that judgment is made thatthe recursive method has been selected, the processing proceeds to StepS236.

In Step S236, the least-N′th-power error coefficient computing unit 234judges whether or not information specifying the exponent N has beeninput, repeats the processing until the exponent N is input, and in theevent that judgment is made that information specifying the exponent Nhas been input by the user operating the operating unit 202, forexample, the processing proceeds to Step S237.

In Step S237, the least-N′th-power error coefficient computing unit 234obtains the coefficient set by an algebraic approach using the basicleast-square error method. In Step S238, the least-N′th-power errorcoefficient computing unit 234 uses a prediction value obtained by thecoefficient set obtained by least-square-error, obtains the coefficientset by a least-N′th-power error input from the operating unit 202corresponding to the exponent N recursively, as described with referenceto Expression (51) through Expression (54), which is stored in thecoefficient memory 235, and the flow returns to Step S231.

Next, with the mapping processing, in Step S241, with a frame of animage serving as output signals corresponding to an image frames servingas current input signals, as a frame of interest, the tap extractingunit 251 of the mapping processing unit 222 takes, of pixels of theframe of interest, those which have not yet been the pixel of interest,in raster scan order, for example, extracts a prediction tap from theinput signals for that pixel of interest, and outputs this to the sum ofproducts computing unit 252.

Then, in Step S242, the sum of products computing unit 252 reads outprediction coefficients of the coefficient memory 235 of the learningunit 221, and executes sum of products computing processing between theprediction tap input from the tap extracting unit 251 and the predictioncoefficient read out from the coefficient memory 235, following theExpression (39). Thus, the sum of products computing unit 252 obtainsthe pixel value (prediction value) for the pixel of interest.Subsequently, the flow proceeds to Step S243, where the tap extractingunit 251 judges whether or not all pixels have been made the pixel ofinterest in the frame of interest, and in the event that judgment ismade that there are some that have not, the flow returns to Step S241,and the same processing is repeated with pixels in the frame of interestwhich have not yet been made the pixel of interest, being made the newpixel of interest, in raster scan order.

Also, in Step S243, in the event that judgment is made that all pixelsin the frame of interest have been made to be the pixel of interest, theflow proceeds to Step S244, where the display unit 203 displays theframe of interest made up of the pixels obtained by the sum of productscomputing unit 252.

The flow returns to Step S241, where the tap extracting unit 251 takesthe next frame as a new frame of interest, and subsequently, the sameprocessing is repeated.

According to the image optimizing processing in FIG. 35, in the mappingprocessing, the user views the image displayed on the display unit 203and in the event this does not match the preferences of the user,operates the operating unit 202 to specify direct method or recursivemethod, and further specify the exponent N for least-N′th-power error,and accordingly in the learning processing, the prediction coefficientsobtained by least-N′th-error are changed, and the image which is theoutput signals obtained by the mapping processing can be made to be thatwhich suits the preferences of the user him/herself.

Now, FIG. 36 illustrates the sum of error of prediction values computedusing a coefficient set of the least-N′th-power norm obtained withleast-N′th-power error, and the sum of error of prediction valuescomputed using a coefficient set of a least-square norm obtained by ageneral least-square error method, with the direct method wherein thevalues of the coefficients a, b, and c, of the weight α_(s) inExpression (50) are changed, by setting the coefficients a, b, and c to,for example, a=40, b=0.1, and c=1. Here, the sum of the square error andcubic error is indicated for the sum of error. Also, in the event thatthe coefficients a, b, and c, are the above-described values, this isequivalent to a case wherein the exponent N of the N′th-power errore^(N) in Expression (48) is greater than 2. In FIG. 36, the sum ofsquare error is 10160281 in the case of coming from the least-squarenorm coefficient set, and is 10828594 in the case of coming from a leastN′th-power norm coefficient set, so the case of the least-square normcoefficient set is smaller than the case of the least N′th-power normcoefficient set. On the other hand, the sum of cubic error is 165988823in the case of coming from the least-square norm coefficient set, and is161283660 in the case of coming from a least N′th-power norm coefficientset, so the case of the least N′th-power norm coefficient set is smallerthan the case of the least-square norm coefficient set.

Accordingly, an image which is output signals wherein the sum of thesquared norm is even smaller can be obtained by performing the mappingprocessing (the sum of products computation in Expression (39)) usingthe coefficient set of the least squared norm, and also, an image whichis output signals wherein the sum of the cubic norm is even smaller canbe obtained by performing the mapping processing using the coefficientset of the least-N′th-power norm obtained using the coefficients a, b,and c, of the above-described values.

Now, in the image optimizing processing in FIG. 35, the exponent N ischanged by the user operating the operating unit 202 (in the directmethod, the coefficients a, b, and c, specifying the exponent N arechanged, and in the recursive method, the exponent N itself is changed)thus setting what sort of least-N′th-power error method for the exponentN is to be employed as the learning norm (learning system) forprediction coefficients (coefficient set). That is, the learningalgorithms for obtaining prediction coefficients themselves are changed.Accordingly, it can be said that the “structure of processing” is beingchanged so that an image suiting the preferences of the user can beobtained.

Next, FIG. 37 illustrates another configuration example of theoptimizing device shown in FIG. 1. With the optimizing device 201 shownin FIG. 37, configurations other than an internal information generatingunit 261 being provided are the same as those of the optimizing device201 shown in FIG. 29, so description thereof will be omitted.

The internal information generating unit 261 reads out, for example,prediction coefficients stored in the coefficient memory 235 as internalinformation of the processing unit 211, converts the information of theprediction coefficients into image signals and outputs this to thedisplay unit 203, where it is displayed.

Next, the image optimizing processing with optimizing device 201 shownin FIG. 37 will be described with reference to the flowchart shown inFIG. 38. The image optimizing device shown in FIG. 38 is also made up oflearning processing and mapping processing, as with the case shown inFIG. 35. With the learning processing, processing the same of that insteps S230 through S238 is performed in steps S250 through S258,respectively.

Further, with the learning processing, following the processing in stepsS255 and S258, the flow proceeds to step S259, for the internalinformation generating unit 261 reads the coefficient set stored in thecoefficient memory 235 as internal information, generates image signalswhich can be displayed based on the values contained in the coefficientset and outputs this to the display unit 203 where it is displayed.

At this time, the image generated by the internal information generatingunit 261 and displayed on the display unit 203 may have a form such as a3-dimensional distribution diagram such as shown in FIG. 39, or a2-dimensional distribution diagram such as shown in FIG. 40, forexample. That is, in FIG. 39, coordinates equivalent to the predictiontap position extracted from the input signals are indicated as positionson an x y plane, as Tap Position (x) and Tap Position (y), withprediction coefficients (Coeff) taking the product of pixel valuesserving as the prediction tap thereof shown at the coordinatescorresponding to each of the prediction tap positions. Also, in FIG. 40,FIG. 39 is represented in the form of contour lines.

Now, let us return to the description of the flowchart in FIG. 38.

Following the processing in Step S259, the flow returns to step S250,and subsequently, the same processing is repeated.

On the other hand, with the mapping processing, processing the same ofthat in steps S241 through S244 in FIG. 35 is performed in steps S261through S264, respectively.

Due to the above processing, the values of the coefficient sets (thecoefficient values) stored in the coefficient memory 235 of theprocessing unit 211 is displayed (present) as internal informationrelating to the processing, so the user operates the operating unit 202while viewing the distribution of the coefficient sets and theprocessing results of the processing unit 211 as output signals so thatan image as output signals suiting the preferences thereof can beobtained, thereby changing the exponent N (in the direct method, thecoefficients a, b, and c, specifying the exponent N are changed, and inthe recursive method, the exponent N itself is changed) thus settingwhat sort of least-N′th-power error method for the exponent N is-to beemployed as the learning norm (learning system) for predictioncoefficients (coefficient set). That is, the learning algorithms forobtaining prediction coefficients themselves are changed, so it can besaid that the “structure of processing” is being changed. Also, in theabove-described example, the coefficient sets are displayed, butarrangement may be made wherein internal information relating toprocessing is displayed such as, for example, whether the currentleast-N′th-power error method is direct or recursive.

FIG. 41 illustrates another configuration example of the optimizingdevice. The optimizing device 301 shown in FIG. 41 is configured of theprocessing unit 311, and optimizes input signals and displays these on adisplay unit 202 based on operating signals input from the operatingunit 202. In the figure, parts corresponding to the above-describedembodiments are denoted with the same reference numerals, anddescription thereof will be omitted in the following, whereverappropriate.

The coefficient memory 321 of the processing unit 311 shown in FIG. 41is basically the same as the coefficient memory 235 shown in FIG. 30,and stores coefficient sets necessary for executing the mappingprocessing by the mapping processing unit 222. The coefficient sets arebasically coefficient sets generated by the learning device 341 shown inFIG. 43 which will be described later (coefficient set serving asinitial values), which are changed by the coefficient changing unit 322as appropriate, and overwritten and stored. Accordingly, as overwritingis repeated, the coefficient sets come to be something different to thatgenerated by the learning device 341. Also, an arrangement may be madewherein the coefficient sets serving as the initial value are stored inunshown memory, so that the storage contents of the coefficient memory321 can be reverted to the initial value coefficient set, correspondingto operations made by the operating unit 202.

The coefficient changing unit 322 reads out coefficient set stored inthe coefficient memory 321 (prediction coefficient), based on operatingsignals input from the operating unit 202, changes the values of theprediction coefficients (prediction coefficient multiplied by theprediction taps) corresponding to the prediction taps, and overwrites onthe coefficient memory 321 again and stores.

Next, the configuration of the coefficient changing unit 322 will bedescribed with reference to FIG. 42. The coefficient read/write unit 331of the coefficient changing unit 322 is controlled by the changeprocessing unit 332, so as to read out the coefficient sets stored inthe coefficient memory 321, and output to the changing processing unit332, while writing the prediction coefficients, of which the values havebeen changed by the changing processing units 332, on the coefficientmemory 321 for storage. The changing processing unit 332 changes theprediction values read out from the coefficient memory 321 by thecoefficient read/write unit 331, based on the operating signals.

Here, a learning device 341 for generating coefficient sets to be storedin the coefficient memory 321 by the learning processing, will bedescribed with reference to FIG. 43. The tutor data generating unit 351of the learning device 341 is the same as the data generating unit 231of the learning device 221 shown in FIG. 30, and generates tutor datafrom image signals which is learning data provided beforehand, andoutputs tutor data to the normal equation generating unit 354. Thestudent data generating unit 352 is the same as the student datagenerating unit 321 shown in FIG. 30, and generates student data fromthe learning data and outputs this to the prediction tap extracting unit353.

The prediction tap extracting unit 353 is the same as the prediction tapextracting unit 233 shown in FIG. 30, wherein, with that the tutor datawhich is to be processed now as the pixel of interest, extracts aprediction tap of the same tap structure as that generated by the tapextracting unit 251 (FIG. 31) making up the mapping processing unit 222shown in FIG. 41 from the student data with regard to that pixel ofinterest, and outputs this to the normal equation generating unit 354.

The normal equation generating unit 354 generates a normal equation ofthe Expression (46) from the tutor data y which is the pixel of interestinput from the tutor data generating unit 351 and prediction taps x₁,x₂, . . . , X_(M). The normal equation generating unit 354 then obtainsthe normal equation in Expression (46) with all tutor data as pixel ofinterest, and outputs the normal equation to the coefficient decidingunit 355. The coefficient deciding 355 solves the input normal equation(the above-described Expression (46)) by the Cholesky method, or thelike, thereby obtaining coefficient sets.

Next, coefficient deciding processing (learning processing) with thelearning device 341 shown in FIG. 43 will be described, with referenceto the flowchart in FIG. 44. In step S271, the tutor data generatingunit 351 generates tutor data from the learning data, and outputs thisto the normal equation generating unit 354, while the student datagenerating unit 352 generates student data from the learning data,outputs this to the prediction tap extracting unit 353, and the flowproceeds to step S272.

In step S272, the prediction tap extracting unit 352 sequentially takestutor data as pixels of interest, extracts prediction taps from thestudent data with regard to the data of interest, outputs this to thenormal equation generating unit 354, and the flow proceeds to step S273.

In step S273, the normal equation generating unit 354 uses the tutordata and prediction tap sets to compute the summation (Σ) of thecomponents of the matrix at the left-hand side of the Expression (46)and the summation (Σ) of the components of the vector of the right-handside thereof, thereby generating a normal equation, which is output tothe coefficient deciding unit 355.

The flow then proceeds to step S274, for the coefficient deciding unit355 solves the normal equation input from the normal equation generatingunit 354 and obtains a coefficient set by so-called least-square error,which is stored in the coefficient memory 321 in step S275.

Due to the above processing, a basic coefficient set (coefficient setserving as an initial value) is stored in the coefficient memory 321.Now, while the above description gives the coefficient set beingobtained by least-square error, this may be a coefficient set obtainedby a another method, and may be a coefficient set obtained by theabove-described least-N′th-power method.

Next, description will be made regarding a changing of the coefficientset by the changing processing unit 332 of the coefficient changing unit322 shown in FIG. 41. Coefficient sets are stored in the coefficientmemory 321 beforehand by the processing of the flowchart in theabove-described FIG. 44, and the coefficient changing unit 322 changesthe prediction coefficients of the coefficient set that is set bycomputation beforehand, based on operating signals input from theoperating unit 202.

For example, in the event that the prediction taps extracted from thestudent data are 7 taps×7 taps (7×7 pixels for horizontal×vertical), atotal of 49 taps, the same number of prediction coefficientcorresponding to the prediction taps exist. That is, in this case, acoefficient set stored in the coefficient memory 321 is a coefficientset made up of 49 prediction coefficients. At this time, with theposition of each of the prediction taps (tap position) as the horizontalaxis (e.g., providing numbers to each of the prediction taps and usingthe value of the number as a horizontal axis), let us say that thedistribution, wherein the coefficient value of the predictioncoefficient multiplied by the prediction tap of the tap positions istaken as the vertical axis, is as shown in FIG. 46. Now, in the eventthat all of the coefficient values of the coefficient set are to bechanged, this requires operations for each of the 49 coefficient values.It is necessary that the values of the coefficient be normalized(wherein each of the coefficients are divided by the total of the valuesof all coefficient) so that gain of the input signals and the outputsignals obtained by processing the input signals with predictioncoefficient is the same, i.e., that the total value of the coefficientbe 1, but it is difficult to operate each of the individual coefficientso that the total is 1.

That is to say, for example, considering raising only the coefficientvalue of the coefficient corresponding to the prediction tap indicatedby the arrow in FIG. 47, the distribution of tap positions andcoefficient values is as shown in FIG. 48. Note that the tap position tshown in FIG. 48 is the tap position indicated by the arrow in FIG. 47.

In this way, raising (increasing) only the coefficient value of thecoefficient corresponding to a particular prediction tap requiresoperations so that the coefficient values of coefficient correspondingto other tap positions are reduced or the like such that the total is 1,but the operations are difficult. Further, operations for making thetotal 1 is similarly difficult in cases of changing the coefficientvalue of coefficient corresponding to a greater number of predictiontaps, as well.

Accordingly, in the event that the operating signals are such that thecoefficient value of a coefficient corresponding to one tap positionindicated by the arrow shown in FIG. 49 is to be changed beyond apredetermined threshold value S11 (the amount of change changes greaterthan a threshold value S11), the change processing unit 332 changes thecoefficient value of other coefficient from the distribution such asshown in FIG. 46 to a distribution such as shown in FIG. 50. That is tosay, the change processing unit 332 changes the coefficient values ofcoefficient corresponding to each of the tap positions, such that thedistribution of coefficient values changes according to a model such aslike a spring, according to the distance from the tap positioncorresponding to the coefficient regarding which the value thereof hasbeen changed. That is, in the event that the distribution of thecoefficient set obtained by learning is a distribution such as shown inFIG. 46, upon operations being made at the operating unit 202 so as toraise the value corresponding to the tap position t as shown in FIG. 50,the change processing unit 332 increases the coefficient value for othervalues at positions close to the tap position t such that the closer theposition is, the greater the changes, and conversely, the coefficientvalues of coefficients corresponding to taps at positions distant fromthe tap position t are changed so that the farther the position, thegreater the decrease is, and further, where the total of coefficientvalues is 1. Here, in the following, the model wherein the distributionchanges in the form of a spring such as, shown in FIG. 50 will be calleda spring model. Further, according to the spring model, in the eventthat operating unit 202 is operated so as to lower the coefficient of acertain tap position t, the coefficient values of positions close to thetap position t are lowered corresponding to the closeness to thatposition, and the coefficient values of positions distant from theposition t are conversely raised according to the distance from thatposition.

Also, in the event that the amount of change of the coefficient value ofthe coefficient is smaller than the predetermined threshold value S11,the change processing unit 332 changes coefficient values having aextreme value the same as the polarity of the coefficient of the tapposition t in the same direction as the coefficient of the tap positiont, and changes coefficient values having a extreme value different tothe polarity of the coefficient of the tap position t in the oppositedirection as the coefficient of the tap position t (shifts coefficientswith the same extreme value as the direction of the extreme value whichthe coefficient that has been operated has, in the same direction as thecoefficient which has been operated, and shifts coefficients with adifferent extreme value as the direction of the extreme value which thecoefficient that has been operated has, in the opposite direction as thecoefficient which has been operated), so as to affect change such thatthe total of coefficient values is 1, maintained the equilibrium of theoverall distribution balance. In the following, the model whereincoefficient values are changed while maintaining the equilibrium of theoverall balance as shown in FIG. 51 will be called the equilibriummodel. Due to effecting such changes, the equilibrium model functionallyapproximates (is equivalent to) a HPF (High Pass Filter) or LPF (LowPass Filter).

Now, a case has been described above wherein the coefficient value witha positive value has been raised in an equilibrium model, but in theevent that the coefficient value with a positive value is lowered, forexample, i.e., change in the negative direction, positive values arechanged in the negative direction, and coefficient values with negativevalues are changed in the positive direction. Further, in the event thatcoefficient values with negative values are raised, coefficient valueswith positive values are changed in the negative direction, and negativecoefficients are changed to a positive direction while in the event thatcoefficient values with negative values are lowered, positivecoefficient values are changed in a positive direction, and negativecoefficient values are changed in the negative direction. Withequilibrium model, in all cases, the values of coefficients are changedin the direction whereby the equilibrium of the overall balance ismaintained.

As described above, in the event that the amount of change of thechanged coefficient value is greater than the threshold value S11, thechange processing unit 332 changes coefficient values corresponding toother taps by the spring model shown in FIG. 50, and in the event thatthe amount of change of the coefficient value is smaller than thethreshold value S11, changes the coefficient value of coefficientscorresponding to other taps by the equilibrium model shown in FIG. 51.This is since in the event that the amount of change of one coefficientis great, changes to maintain the overall balance are unnatural from theperspective of overall balance of each of the coefficient values, sothis spring model is employed, and in the event that the amount changesare small, the effect that the change in coefficients has on the overallbalance is small, so change is made so as to maintain overall balance.

Now, models for changing the coefficients other than the changedcoefficient values by operating unit 202 being operated are notrestricted to these, and any is sufficient as long as the total ofcoefficient values is 1 as a whole. Also, in the above-described case,models for changing other coefficient are switched over according to themagnitude of amount of change of a coefficient which is changed by theoperating unit 202 being operated, but the model for changing othercoefficients may be fixed.

Next, the image optimizing processing of the optimizing device 301 shownin FIG. 41 will be described with reference to the flowchart shown inFIG. 52. Now, this image optimizing processing is made up of coefficientchanging processing and mapping processing. The mapping processing isthe same as the mapping processing described with FIG. 35 and FIG. 38,so only the coefficient changing processing will be described here.

In step S291, the changing processing unit 332 of the coefficientchanging unit 322 (FIG. 42) judges whether or not operating signals foroperating coefficient values have been input from operating unit 202.That is, in the event that the user deems the image displayed on thedisplay unit 203 to be something matching his/her preferences, mappingprocessing is executed with a coefficient set stored in the coefficientmemory 321 (FIG. 41) here, but in the event that judgment is made thatthis does not match preferences, operations are performed to change thecoefficient set stored in the coefficient memory 321 to be used formapping processing.

For example, in step S291, in the event that judgment is made thatoperating signals for operating coefficients have been input, i.e., inthe event that operating unit 202 has been operated so as to change thecoefficient value of one of the coefficient stored in-the coefficientmemory 321, the flow proceeds to step S292.

In step S292, the change processing unit 332 controls the coefficientread/write unit 331 to read out coefficient sets stored in thecoefficient memory 321, and the flow proceeds to step S293. In stepS293, the change processing unit 332 judges whether or not thecoefficient value input as operating signals is a change equal to orgreater than a predetermined threshold value S11 in comparison to thevalue contained beforehand in the coefficient set. For example, in theevent that judgment is made in step S293 that the change between thevalue input as operating signals and the value in the coefficient setstored in the coefficient memory 321 is equal to or greater than thethreshold value S11, the processing then proceeds to step S294.

In step S294, the change processing unit 332 changes the values of thecoefficient contained in the coefficient set with the spring model asshown in FIG. 50, and the processing then proceeds to step S295.

On the other hand, in the event that judgment is made in step S293 thatthe change between the value input as operating signals and the value inthe coefficient set stored in the coefficient memory 321 is not equal toor greater than the threshold value S11, the processing then proceeds tostep S296.

In step S296, the change processing unit 332 changes the value of thecoefficients contained in the coefficient set with the equilibrium modelas shown in FIG. 51, and the processing then proceeds to step S295.

In step S295, the change processing unit 332 controls the coefficientread/write unit 331 so as to overwrite and store the changed coefficientset values to the coefficient memory 321, the processing returns to stepS291, and subsequently, the same process is repeated.

In the event that judgment is made in step S291 that the operatingvalues have not been operated, i.e., in the event that the user hasjudged that the image displayed on the display unit 203 matches his/herpreferences, the flow returns to step S291, and subsequently, the sameprocessing is repeated.

Due to the above coefficient changing processing, the user can changecoefficient sets used for mapping processing, thereby executing optimalprocessing for the user. Now, changing the values of the coefficients ofthe coefficient set means that the “contents of processing” of themapping processing by the mapping processing unit 311 are changed.

Also, with the coefficient changing processing in FIG. 52, in the eventthat the magnitude of change of the coefficient is equal to or greaterthan a predetermined threshold value S11, all coefficient values in thecoefficient set are changed with the spring model according to the valueof the coefficient that has been operated, and in the event that this issmaller than the threshold value S11, all coefficient values in thecoefficient set are changed with the equilibrium model, so thatalgorithm for changing the coefficient set changes. Accordingly, at theprocessing unit 311 of optimizing device 301 shown in FIG. 41 as well,the “contents of processing”, and further, the “structure of processing”is also changed following user operations, with optimal signalprocessing for the user being performed thereby.

Also, as described above, in the event that the coefficients set storedin the coefficient memory 321 is obtained by least-N'th-power error, anarrangement may be made wherein, for example, coefficients setscorresponding to multiple exponents N are stored in the coefficientmemory 321 beforehand, with the coefficient changing unit 322 changingthe coefficient set corresponding to the specified exponent N, followingoperating signals from the operating unit 202 based on user operations.In this case, the coefficient sets stored in the coefficient memory 321are changed into those which are generated by the least-N'th-power errormethod corresponding to the exponent N input from the operating unit 202based on user operations i.e., changed to the coefficient set generatedby a different coefficient set generating algorithm, so it can be saidthat the “structure of processing” has been changed.

Next, an embodiment wherein an internal information generating unit 371is provided to optimizing device 301 shown in FIG. 41 will be describedwith reference to FIG. 53. Note that in FIG. 53, this is the same as tooptimizing device 301 shown in FIG. 41 except for the point that theinternal information generating unit 371 has been provided.

The internal information generating unit 371 reads out coefficient setstored in the coefficient memory 321 for example, as internalinformation of the processing unit 311, converts the set to imagesignals which can be displayed on the display unit 203, then outputs tothe display unit 203, and displays.

Next, the image optimizing processing of optimizing device 301 shown inFIG. 53 will be described with reference to the flowchart in FIG. 54.This image optimizing processing is also made up of coefficient changingprocessing and mapping processing in the same way as the imageoptimizing processing which the optimizing processing 301 shown in FIG.41 performs, but the mapping processing is the same as the mappingprocessing described with FIG. 35 and FIG. 38, so only the coefficientchanging processing will be described here.

With the coefficient changing processing, processing the same of that insteps S291 through S295 in FIG. 52 is performed in steps S311 throughS315, respectively.

In step S315, as with the case in step S295 in FIG. 52, following thepost-conversion coefficient set being stored in the coefficient memory321, flow proceeds to step S317, where the internal informationgenerating unit 371 reads out the coefficient values of the coefficientset stored in the coefficient memory 321, converts these to imagesignals which can be displayed on the display unit 203, outputs to thedisplay unit 203, and displays. At this time, the display unit 203 maydisplay the coefficient values of coefficient sets with a form such as a3-dimensional distribution diagram such as shown in FIG. 39, or a2-dimensional distribution diagram such as shown in FIG. 40, forexample.

Following the processing step is 317, the flow returns to step S311, andsubsequently, the same processing is repeated.

According to the coefficient changing processing in FIG. 54, the valuesof the coefficient set stored in the coefficient memory 321 aredisplayed as internal information, so the user can operate the operatingunit 202, while viewing the coefficient set, so as to obtain acoefficient set which executes processing that is optimal for the user.

Also, with the sum of products computing unit 252 of the mappingprocessing unit 222 (FIG. 31), output signals can be obtained by acomputing a higher order expression of an order of two or higher,instead of the linear expression in Expression (39).

Next, a configuration example of an optimizing device 401 which extractstelop portions from image signals as input signals will be describedwith reference to FIG. 55.

The features detecting unit 411 of optimizing device 401 detects twotypes of features that have been specified with regard to each of thepixels of the image signals serving as input signals, based on operatingsignals input from an operating unit 402, and outputs be detectedfeatures information to a processing deciding unit 412. Also, thefeatures detecting unit 411 stores the image signals serving as inputsignals in an internal buffer 421 until the telop is extracted from theinput image signals, and outputs the image signals to the processingunit 413. The operating unit 402 is the same as the operating unit 202shown in FIG. 41 and FIG. 53. Note that the features detecting unit 411is not restricted to only detecting two types of features that have beenspecified with regard to each of the pixels of the image signals servingas input signals, and may, for example, simultaneously detect multipletypes of features, and output two types of specified features fromthese, or simultaneously detect two or more types of features andsimultaneously output these.

The processing deciding unit 412 decides processing which a laterprocessing unit 413 will perform on the image signals, in increments ofpixels, for example, based on the features input from the featuresdetecting unit 411, and outputs the decided processing contents to theprocessing unit 413.

The processing unit 413 subjects the image signals serving as inputsignals read out from the buffer 421 to the processing of the processingcontents input from the processing deciding unit 412 in increments ofpixels, outputs this to the display unit 403, and displays.

Next, the configuration of the features detecting unit 411 will bedescribed with reference to FIG. 56. The buffer 421 of the featuresdetecting unit 411 temporarily stores the image signals which are theinput signals, and supplies these to the processing unit 413. Thefeatures extracting and 422 extracts the two types of features selectedby the features selecting unit 423 from the image signals serving as theinput signals, and outputs these to the processing deciding unit 412.The feature selecting unit 423 supplies information for specifying thefeatures to be extracted from the input signals, to the featuresextracting unit 422, based on operating signals input from operatingunit 402. Features which can be selected are, for example, luminancevalues for each of the pixels of the image signals, Laplacian, Sobel,inter-frame of indifference, inter-field difference, the backgrounddifference, and values obtained from the features within a predeterminedrange (sum, average, dynamic range, maximum value, minimum value,central value, or dispersion, etc.), and so forth, but may be otherfeatures as well.

Next, the configuration of the processing deciding unit 412 will bedescribed with reference to FIG. 57. The features recognizing unit 431of the processing deciding unit 412 recognizes the multiple types offeatures input from the features detecting unit 421, and outputs, alongwith information indicating the type of features recognized, thefeatures themselves, to the processing contents deciding unit 432. Theprocessing contents deciding unit 432 decides the contents of processingset for each of the features beforehand stored in the processingcontents database 433, based on the information indicating the type offeatures input from the features recognizing unit 431, and the featuresthemselves, and outputs decided processing contents to the processingunit 413.

Next, the configuration of the processing unit 413 will be describedwith reference to FIG. 58. The processing content recognizing unit 441of the processing unit 413 recognizes the processing contents input fromthe processing deciding unit 412, and instructs the processing executingunit 442 to execute processing that has been recognized. The processingexecuting unit 442 subjects the input signals input via the buffer 421to the specified processing, based on the instructions for each pixelfrom the processing contents recognizing unit 441, converts these intoimage signals which can be displayed on the display unit 202 and outputsto the display unit 403, and displays.

Note that the optimizing device 401 shown in FIG. 55 is for extracting atelop from image signals, so the processing contents are processingwhether or not to extract each pixel as a telop portion (to display ornot), but may be made to subject the image signals to other processingas well, and this may be processing wherein input signals are outputwithout change with regard to pixels recognized as being a telopportion, while pixels recognized as being other than telop portions arenot output. Also, the image input as input signals will be described asa single frame still image, to facilitate description. However, theoptimizing device 401 in FIG. 55 is also applicable to moving images.

Next, the telop extracting optimizing processing with the optimizingdevice shown in FIG. 55 will be described, with reference to theflowchart in FIG. 59.

In step S331, the features extracting unit 422 of the features detectingunit 411 judges whether or not two types of features have been selectedby the features selecting unit 423, and repeats the processing untilthese are selected. That is, the processing in step S331 is repeateduntil information indicating the features selected by the featuresselecting unit 423 is input to the features extracting unit 422 based onoperating signals corresponding to the type of features input by theuser operating the operating unit 402. For example, in the event thatjudgment is made that information for selecting the features has beeninput from the features selecting unit 423, i.e., in the event thatjudgment is made that the user has operated operating unit 402 andselected two types of features, the processing proceeds to step S332.

In step S332, the features extracting unit 422 extracts the two types ofselected features from the image signals which are the input signals,for each pixel, and outputs these to the processing deciding unit 412.At this time, the image signals which are the input signals are storedin the buffer 421.

In step S333, the processing deciding unit 412 decides the processingcontents based on the two types of input features, for each of thepixels, and outputs these to the processing unit 413. In more detail,the features recognizing unit 431 identifies the two types of inputfeatures, and outputs the identified types of features and the featuresthemselves, to the processing deciding unit 412. Further, the processingdeciding unit 412 decides the processing contents from the two types offeatures input for each of the pixels. More specifically, a table calledan LUT (Look Up Table) correlating, for each of a combination of twoarbitrary types which are a feature A and feature B (feature A, featureB), each value of feature A and feature B with processing contentsregarding pixels having the features A and B (in this case, informationwhether or not this is a telop), is stored in the processing contentsdatabase 433, with the processing contents deciding unit 432 makingreference to the LUT based on the combination of the (feature A, featureB) of the pixel of interest which is to be now processed, and decidesthe corresponding processing, i.e., whether or not to process as atelop, and outputs this to the processing unit 413. Also, this LUT isgenerated by extracting multiple features from telop-only imagesbeforehand, for example, and correlating with information that thecombination is a telop. Details of the LUT will be described later.

In step S334, the processing unit 413 processes the image signalsserving as input signals input via the buffer 421, following theprocessing contents input from the processing deciding unit 412,converts these into image signals which can be displayed on the displayunit 403, outputs to the display unit 403, and displays. In furtherdetail, the processing contents recognize unit 441 of the processingunit 413 recognizes the processing contents input from the processingdeciding unit 412, and instructs the processing executing unit 442 toexecute processing that has been decided with regard to thecorresponding pixels. The processing executing unit 442 reads out theimage signals serving as a input signals that are stored in the buffer421, executes processing corresponding to each of the pixels, convertsthese into image signals which can be displayed on the display unit 403,outputs to the display unit 403, and displays.

In step S335, the features detecting unit 411 judges whether or not atelop is deemed to be extracted. That is to say, in the event that theuser views the image displayed on the display unit 403 and does notjudge that a telop is being extracted, the operating unit 402 isoperated so that the combination of features is changed again and telopextraction processing is attempted. Upon operating signals from theoperating unit 402 corresponding to this operation being input, theprocessing returns to step S331, and the subsequent processing isrepeated.

On the other hand, in the event that judgment is made according to thesubjectivity of the user that a telop is being extracted, operatingsignals indicating completion of the processing are input tothe-features detecting unit 421 by the user operating the operating unit402, and the processing ends at this time.

That is to say, the processing of steps S331 through S335 is repeateduntil the user can judge that a telop is being extracted by viewing theimage displayed on the display unit 403, due to the above-describedprocessing, whereby a combinations of features optimal for the user canbe set, and telops can be extracted from image signals serving as inputsignals. Now, in the above described, two types of features are used todecide the processing contents, but an arrangement may be made whereinthe processing contents are decided by features of the number of typesother than that number. Also, an arrangement may be made wherein, in theprocessing in step S331, combinations of multiple features aresequentially switched over in a predetermined order by operating signalscorresponding to a predetermined operations made by the user at theoperating unit 402 (e.g., button operations instructing up or down,etc.) thereby allowing the user to switch over and input the featureswithout particularly giving consideration to the type of features.

In the above processing, the type of features detected by the featuresdetecting unit 411 is changed at the processing unit 413 according tooperations of the operating unit 402 by a user, so that the telop isdetected. The changing of type of features detected at the featuresdetecting unit 411 indicates change in the processing contents decidingalgorithm at the processing deciding unit, so it can be said that at thefeatures detecting unit 411 as well, the “structure of processing” ischanged.

Also, as described above, detection of various types of features can bemade at the features detecting unit 411, but some of the featuresrequire setting of parameters such as filter coefficients and the likein order to detect, such as Laplacian, for example. Parameters fordetecting features can be made changeable corresponding to operations ofoperating unit 402, but according to this parameter changing, the typesof features themselves being detected at the features detecting unit 411do not change, but the values of the features being detected do change.Accordingly, it can be said that the change of parameters for detectingthese features is a change of the “contents of processing” of thefeatures detecting unit 411.

Next, the configuration of an optimizing device 501 wherein theoptimizing device 401 shown in FIG. 55 is provided with an internalinformation generating unit 511, with reference to FIG. 60. Theoptimizing device 501 shown in FIG. 60 is basically the same inconfiguration as the optimizing device 401 shown in FIG. 55, other thanthe internal information generating-unit 511 being provided.

The internal information generating unit 511 of the optimizing device501 extracts, as internal information, features collecting informationoutput from the features selecting unit 423 of the features detectingunit 411, for example, and displays the type of features currentlyselected on the display unit 403.

Now, the telop extracting optimizing processing performed by theoptimizing device 501 will be described with reference to flowchart inFIG. 61.

Note that this processing is basically the same processing as the telopextraction optimizing processing by the optimizing device 401 shown inFIG. 56, described with reference to the flowchart in FIG. 59, anddiffers in the point that processing for displaying the informationindicating the type of features that has been selected, has been added.

That is to say, in step S341, the features extracting unit 422 of thefeatures detecting unit 411 judges whether or not two types of featureshad been selected by the features selecting unit 423, and repeats theprocessing until they are selected. For example, in the event thatjudgment is made that information for selecting features has been inputfrom the features selecting unit 423, i.e., in the event that judgmentis made that the user has operated the operating unit 402 and selectedtwo types of features, the processing proceeds to step S342.

In step S342, the internal information generating unit 511 extractsinformation indicating the two types of features that have been selectedfrom the features selecting unit 423, and displays the name of the typesof the two features that have been selected, on the display unit 403.

Subsequently, processing that is the same as that in steps S332 throughS335 is performed in steps S343 through S346, respectively.

According to the processing in FIG. 61, the types of features currentlyselected, which is internal information relating to the processing ofthe features detecting unit 411, is displayed (presented), so the usercan set a combination of features optimal for precisely extracting telopfrom the image signals serving as input signals, while understanding thetypes of features currently being selected.

Now, the internal information generating unit 511 is capable of, forexample, generating a distribution of values of two types of featuresdetected by the features detecting unit 411 with regard to each of thepixels, as internal information, and displaying this on the display unit403, as indicated in the later-described FIG. 65 and FIG. 67.

Also, as described above, in the event of changing the parameters fordetecting features according to operations of operating unit 402, theinternal information generating unit 511 can display (present) theparameters on the display unit 403, as internal information.

Next, a configuration example of an optimizing device 601 arranged so asto be provided with an internal information generating unit 611 whichgenerates internal information from the processing deciding unit 412,instead of the internal information generating unit 511 shown in FIG.60, will be described with reference to FIG. 62.

The optimizing device 601 shown in FIG. 62 is configured in the same wayas the optimizing device 501 in FIG. 60 other than the internalinformation generating unit 611 has been provided instead of theinternal information generating unit 511 shown in FIG. 60.

The internal information generating unit 611 generates, as internalinformation, a distribution diagram (e.g., FIG. 65, FIG. 67) of pixelssubjected to a telop extraction at the time of two types of featuresserving as axes, and pixels not subjected to telop extraction, based onthe processing contents decided by the processing contents deciding unit432 of the processing deciding unit 412, and the two types of featuresactually extracted, which is displayed on the display unit 403.

Next, the telop extraction optimizing processing by the optimizingdevice 601 shown in FIG. 62 will be described with reference to theflowchart in FIG. 63.

Note that this processing is basically the same processing as a telopextraction optimizing processing performed by the optimizing device 501shown in FIG. 60, which has been described with reference to theflowchart in FIG. 61, except for the point that processing wherein adistribution regarding whether or not pixels have been subjected totelop extraction with the two types of selected features as axes, isdisplayed.

That is to say, in step S351, the features extracting unit 422 of thefeatures detecting unit 411 judges whether or not two types of featureshave been selected by the features selecting unit 423, and theprocessing is repeated until these are selected. For example, in theevent that judgment is made that information has been input from thefeatures selecting unit 423 for selecting features, i.e., in the eventthat judgment is made that the user has operated the operating unit 402and selected two types of features, the processing proceeds to stepS352.

In step S352, the features extracting unit 422 extracts two types ofselected features from the image signals serving as input signals foreach of the pixels, and outputs these to the features deciding unit 412.At this time, the image signals which are the input signals are storedin the buffer 421.

In step S353, the processing deciding unit 412 decides processingcontents for each of the pixels based on input two types of features,and outputs this to the processing unit 413.

In step S354, the processing unit 413 processes the image signals whichare the input signals read out from the buffer 421 following theprocessing contents input from the processing deciding unit 412,converts these to image signals that can be displayed on the displayunit 403, outputs to the display unit 403, and displays.

In step S355, the internal information generating units 611 generates,as internal information, a distribution diagram plotting the processingcontents decided by the processing contents deciding unit of theprocessing deciding unit 412, with the two types of features as axes,and displays this on the display unit 403.

In step S356, the features detecting unit 411 judges whether or not atelop is being extracted. In step S356, in the event that operatingsignals corresponding to user operations are being input from theoperating unit 202, and it is deemed that there is no telop extraction,the processing returns to step S351, and the subsequent processing isrepeated.

On the other hand, in the event that in step S356, operating signalsindicating ending of the processing are input to the features detectingunit 421 by the user operating the operating unit 402, the processingends.

That is, let us say that image signals which are input signals such asshown in FIG. 64, are input, for example. In FIG. 64, “TITLE ABC” isdisplayed as a telop in the center of the diagram, as to the backgroundimage (here, the portion which is not a telop).

In step S355, the internal information generating unit 611 displays thedistribution indicating whether or not there has been extraction as atelop, with the two types of features detected from the image signalssuch as shown in FIG. 64 as respective axes, as a 2-dimensionaldistribution diagram such as shown in FIG. 65, for example.

In the example in FIG. 65, Laplacian and into-frame of difference arethe two types of features that have been selected, with the circles inthe figure indicating pixels that have been extracted as a telop, andthe X's indicating pixels regarding which there has been no telopextraction. With the example in FIG. 65, pixels with telop extractionand pixels with no telop extraction are borderless in the distribution(the distribution between telop pixels and non-telop pixels is notdivided). In the case of such a distribution, the telop is often notbeing extracted from the background image, and as shown in FIG. 66 forexample, is in a state with borders 621 and 622 being formed around thetelop portions, instead of the telop portions themselves being borders.

In such cases, the user will judge that the telop is not beingextracted, and the processing in steps S351 through S356 is repeated. Inthe event that the selected features due to repetition of thisprocessing are Laplacian sum (17 pixels×17 pixels) (the sum of eachLaplacian of the pixels within the 17 pixel×17 pixel range centered onthe pixel of interest) and luminance DR (the dynamic range of pixelswithin the 17 pixel×17 pixel range centered on the pixel of interest,for the luminance value of each pixel)r distribution diagram isgenerated such as shown in FIG. 67. In this case, in FIG. 67, thedistribution of pixels extracted as telop and the distribution of pixelsnot extracted as telop are visually separated. This indicates that thetelop in the input image and other parts are divided into distributionin the event that the combination of the Laplacian sum and luminance DRis selected as features. Upon such a distribution being obtained, theLaplacian sum and luminance DR which are the features detected from eachof the pixels are subjected to a threshold processing and the like,thereby allowing the telop portion to be precisely extracted, as shownin FIG. 68.

Thus, due to the above-described processing, the processing in stepsS351 through S356 is repeated until the user can judge by viewing theimage displayed on the display unit 403 and the 2-dimensionaldistribution with the two selected types of features as axes that thetelop is being extracted, so that the distribution of pixels detected tobe telop from the features serving as internal information relating tothe processing of the processing deciding unit 412, and backgroundpixels, is displayed, whereby the user can operate the operating units402 so as to precisely extract the telop from the image signal which arethe input signals, while understanding the distribution of pixelsdetected to be a telop from the features, and background pixels, therebysetting a combination of features optimal for the user.

Next, a description will be made regarding the configuration of anoptimizing device 701 provided with a processing deciding unit 711instead of the processing deciding unit 412 of optimizing device 601shown in FIG. 62, with reference to FIG. 69. With optimizing device 701,configurations are the same as those of optimizing device 601 shown inFIG. 62, except that the processing deciding unit 711 is provided instep of the processing deciding unit 412.

The processing deciding unit 711 changes the contents of the LUT in theprocessing contents database 433 based on operating signals from theoperating unit 702 (the same as the operating unit 402), and alsodecides processing for the later processing unit 413 to perform on imagesignals based on features input from the features detecting unit 411,and outputs the processing contents to the processing unit 413.

The processing deciding unit 711 will next be described with referenceto FIG. 70. The basic configuration is the same as the processingdeciding unit 412 shown in FIG. 50, but processing contents decidingunit 721 is provided instead of the processing contents deciding unit432. The processing contents deciding unit 721 changes the LUT whereinis decided the processing contents for each of the combinations of twotypes of features stored in the processing contents database 433, basedon operating signals input from the operating unit 702. In more detail,the LUT is set such that the features of pixels in an area specified asa telop by the operating unit 202 what is deemed to be a telop in thestate of the image signals which are the input signals being displayedwithout change, and other areas are processed as being other than thetelop.

Following the LUT been changed, the processing contents deciding unit721 decides the processing contents set beforehand for each of thefeatures that are stored in the processing contents database 433, basedon the information for identifying the features input from the featuresrecognizing unit 431, and the features themselves, and outputs thedecided processing contents to the processing unit 413.

Next, the telop extraction optimizing processing by the optimizingdevice 701 shown in FIG. 69 will be described with reference to theflowchart in FIG. 71.

In step S361, image signals which are the input signals are displayed onthe display unit 403 without change. More specifically, the buffer 421of the features detecting unit 411 receive the image signals serving asinput signals and store these, and the processing unit 413 reads out theimage signals which are the input signal stored in the buffer 421without change, and outputs these to the display unit 403 without anyprocessing, where they are displayed.

In step S362, the processing contents deciding unit 721 of theprocessing deciding unit 711 judges whether or not the telop andbackground have been instructed from the operating unit 702. That is,let us say that, for example, unprocessed image signals such as shown inFIG. 72 are displayed in the processing in step S361. At this time, theuser operates a pointer 741 via the operating unit 702 (by dragging,clicking, etc.), to specify a general telop portion by a range 742 orthe like, and as shown in FIG. 73, repeats the processing until thetelop portions 752 and the background portion 751 outside the range 742in FIG. 72 are specified, and that the point that the telop and thebackground portion of specified, the specified perspective correspondingpixel positions are stored in built-in memory (not shown), and theprocessing proceeds to step S363.

In step S363, the features selecting unit 423 of the features detectingunit 421 judges whether or not operating signals for selecting thepredetermined two types of features have been input from the operatingunit 202, the processing is repeated until the predetermined two typesof features are specified, and upon the predetermined two types offeatures being selected, the processing proceeds to step S364.

In step S364, the features extracting unit 422 extracts the two types offeatures selected from the input signals, based on information forselecting the types of features a input to the features selecting unit423, and outputs these to the processing deciding unit 711.

In step S365, the internal information generating units 611 generates a2-dimensional distribution diagram with the two types of features asaxes thereof, based on the two types of features input to the processingdeciding unit 711 and the information of pixel position specified forthe telop and the background, and in step S366, displays the2-dimensional distribution diagram on the display unit 403. In furtherdetail, the features recognizing unit 431 of the processing decidingunit 711 recognizes the type of features, outputs information indicatingthe type, and the features themselves, to the processing contentsdeciding unit 721, and the processing contents deciding unit 721 outputsinformation indicating the pixel positions specified for the telop andbackground to the internal information generating 611, in addition tothe information indicating features and the types thereof, so that theinternal information generating unit 611 generates 2-dimensionaldistribution diagram of the features such as shown in FIG. 74, forexample, based on the information indicating the pixel positionsspecified for the telop and background. That is to say, with the examplein FIG. 74, an example is illustrated wherein Laplacian and inter-frameddifference are selected as features, with the circles in the figureindicating telop pixels, and the X's in the figure indicating pixelspecified as being the background. With the 2-dimensional distributiondiagram in FIG. 74, for example, with regard to a particular pixel, theLaplacian is detected at a value of X, the inter-frame difference isdetected at a value of Y and in the event that that pixel has beenspecified as telop, a circle is displayed at the position of (X, Y) onthe 2-dimensional distribution, while in the event that background hasbeen specified, an X is displayed at that same position.

In step S367, the processing contents deciding unit 721 judges whetheror not operating signals indicating that judgment has been made that thetelop and background have been separated have been input. That is, forexample, in the case of a 2-dimensional distribution such as shown inFIG. 74, the distribution of circles indicating the telop and thedistribution of X's indicating the background cannot be said to becompletely separated, so as a matter of course, the telop cannot beexpected to be extracted on the display screen, and in many cases thebackground and telop are in the state of not being separated such as, asshown in FIG. 73 for example, a border 753 is generated separate to thebackground portion 751 and the telop portions 752. In such cases whereinthe user judges that the telop is not extracted, at the point of theuser attempting to change the features again, the operating unit 702outputs operating signals indicating that separation has not beenachieved to the processing contents deciding unit 721 of the processingdeciding unit 711, in accordance with operations made by the user. Inthis case, in step S367, judgment is made that there have not beenoperating signals indicating judgment that the telop and background havebeen separated, so the processing returns to step S363, and thesubsequent processing is repeated. This processing means that two typesof features are selected again.

Also, in the event that as shown in FIG. 75, for example, the circlesindicating the telop portion and the X's indicating the background areseparated to a certain extent by the processing in step S366, theoperating unit 702 is operated by the user to output operating signalsindicating that separation has been achieved, to the processing contentsdeciding unit 721 of the processing deciding unit 711. In this case,judgment is made that operating signals indicating a judgment that thetelop and background have been divided has been input in step S367, andthe processing proceeds to step S368.

In step S368, the processing contents deciding unit 721 judges whetheror not the telop portion has been instructed on the 2-dimensionaldistribution. That is, as shown in FIG. 75, judgment is made whether ornot operating signals specifying the range 761, for example, by theoperating unit 702, has been input as the range where a great number ofcircles indicating the telop are distributed on the displayeddistribution, with the pointer 741, the processing is repeated until therange specified, and upon judgment being made that this has beenspecified, the processing proceeds to step S369.

In step S369, the processing contents deciding unit 721 changes theprocessing contents: the LUT indicating whether or not there has beentelop extraction corresponding to combinations of the features, based onoperating signal specified in the range 761 shown in FIG. 75 that hasbeen input by the operating unit 702, and decides the processing contextaccording to the changed LUT which are output to the processing unit for13, while the processing unit 413 follows the input processing contentsto extract the telop from the image signals serving as the input signalthat had been input via the buffer 421, and to display these owned thedisplay unit 403. In further detail, the processing contents decidingunit 721 updates the LUT in the processing contents database 433 so asto extract the combination of two types of features corresponding to thepixels distributed in a specified range as telop, based on informationindicating a range on the 2-dimensional distribution such as shown inFIG. 75, that has been input from the operating unit 702, and furtherdecides the processing contents for each pixel following the updatedLUT, which are output to the processing unit 413.

In step S370, the processing contents deciding unit 721 judges whetheror not judgment has been made that the telop has been extracted. Thatis, judgment is made whether the user judges the telop to be extractedor judges the telop to not be extracted. For example, as shown in FIG.76, in the state that the output images displayed on the display unit403 by the processing in step S369, the boundaries 771 and 772 betweenthe telop portion and background portion are not the telop itself, so itcannot be said that the telop is completely extracted. Accordingly, inthe event that the telop is judged to be not extracted from theperspective of the user, the user operates the operating unit 702 toinput operating signals indicate that the telop has not been extracted.Upon receiving these operating signals, the processing contents decidingunit 721 of the processing deciding unit 711 makes judgment in step S730that the telop is not extracted, and the processing proceeds to stepS731.

In step S371, judgment is made regarding whether or not to re-specifythe range for the telop on the 2-dimensional distribution, and in theevent that judgment is made to that not re-specifying the range for thetelop on the 2-dimensional distribution has been selected, theprocessing returns to step S368, and the subsequent processing isrepeated.

On the other hand, in step S370, in the event that re-specifying therange for the telop has been selected by the processing in step S371,for example, a range 781 wherein the portion where the distribution ofcircles indicating the telop exists has been narrowed down as comparedto the range 761 in FIG. 75 (a range narrowing down the portioncontaining a great number of circles extracted more as telop), as shownin FIG. 77, is set in step S368. That is, in FIG. 75, even in the eventof setting the range 761 on the features distribution as a telopportion, consequently, the boundaries 771 and 772 between the telopportion and background portion are not the telop itself, as shown inFIG. 76, so the telop is not completely extracted.

Accordingly, the user operates the operating unit 702 to set a range 781which is narrower than the range 761, as a range over the featuresdistribution (as a range of a portion to specify as a telop portion).Setting the range thus narrows down the range on the featuresdistribution to be extracted as a telop, that is to say, the backgroundportion is more readily eliminated. However, ignoring the range to beextracted as telop too much causes the telop itself to not be readilyextracted, so the user searches for an optimal telop extractions stateby repeating such operations while viewing the extracted telop.

Upon the state being that such as shown in FIG. 78, for example, whereinthe telop is extracted, and in the event that the user judges that thetelop is extracted in step S370, the user operates the operating unit702, and operating signals indicating that jet and has been made thatthe telop has been extracted are input to the processing contentsdeciding unit 721 of the processing deciding unit 711, so judgment ismade that the telop is extracted, and the processing ends.

Also, in the event that judgment is made in step S371 that this does notspecified the range of the telop, the processing returns to step S363,and subsequent processing is repeated.

Due to such processing, first, the user selects two types of features ofthe telop and background on the image signals which are the inputsignals, and, viewing how the selected telop and background form a2-dimensional distribution with the selected features as the two axesthereof, changes (narrows down) the range of the telop on the2-dimensional distribution, or changes the two selected features, andfurther views that extracted telop, and repeats this processing untilthe state is that preferred by the user, thereby a realizing telopextraction processing matching the preferences of the user. In the eventthat judgment is made in the processing in step S362 that the telop andbackground have been instructed, it is sufficient that instructions ofthe range of telop and background serves to subsequently select twotypes of features or generate information serving as a template fornarrowing down the telop on the 2-dimensional distribution, so the rangecan be very general.

In the above processing, division is made into telop or backgroundprocessing, according to combinations of two types of features selectedby the user, so this means that the “contents of processing” have beenchanged by user operations.

Also, with the above processing, the type of features are determined bya two types of features being specified by the operating unit 402, butan arrangement may be made such as shown in FIG. 79, for example,wherein the combination of the two types of features can be changed by asending instructions of up or down as operating signals by apredetermined operating button on the operating unit 702. That is, anarrangement may be made wherein, as an initial state, processing isexecuted by a combination of features a and b indicated in state A, thennext, instructing down executes processing by the combination offeatures b and c indicated in the state B, and further instructing downexecutes processing with a combination of the features c and d indicatedin the state C, wherein, in the event of specifying up in the state C,this returns to the state B, and in the event of specifying up in thestate B, this returns to the state A. Due to such an arrangement, theuser can change the features one after another without giving noparticular consideration to the type of features, so the combination offeatures for extracting telop can be effectively narrowed down.

Next, description will be made regarding a configuration of anoptimizing device 801 provided with the features detecting unit 811capable of generating new features from already-existing features,instead of the features detecting unit 411 of the optimizing device 401in FIG. 55, with reference to FIG. 80. In FIG. 80, the configuration isthe same except for providing the features detecting unit 811 instead ofthe features detecting unit 411 of the optimizing device 401 in FIG. 55.

Further, the operating unit 802 is the same as the operating unit 402.

The configuration of the features detecting unit 811 shown in FIG. 80will be described with reference to FIG. 81. With the features detectingunit 811, the buffer 421, and the features extracting unit 422 are thesame as the features detecting unit 411 shown in FIG. 56. This featureselecting unit 821 controls the features extracting unit 422 based onoperating information specifying the features input from the operatingunit 802, to extract two types of features that have been specified outof the features provided beforehand, and output these to the processingdeciding unit 413, or output features stored in the features database823 beforehand, to the processing deciding unit 413. In more detail, thefeatures database 823 stores features information regarding the types offeatures and the detection methods for the features. The featuresextracting unit 422 reads out features information corresponding to thetypes of features selected by the features selecting unit 821 from thefeatures database 823, and detects the selected features from the inputsignals, following the features detecting method recorded in thefeatures information.

Features information provided beforehand are luminance values for eachof the pixels of the image signals, Laplacian, Sobel, inter-framedifference (e.g., inter-frame difference of a pixel of interest (x, y)is represented by f_(s)(x, y)=f₀(x, y)+f₁(x, y), wherein f₀(x, y)indicates the current pixel of interest, while f₁(x, y) represents apixel from one frame back on the same spatial position), inter-fielddifference, background difference, derivative values (e.g., thederivative value f_(b)(x, y) of a pixel of interest (x, y) isrepresented by f_(b)(x, y)=4×f₀(x, y)−f₀(x−1, y)−f₀(x+1, y)−f₀(x,y+1)−f₀(x, y−1), wherein f₀(x, y) represents the current pixel ofinterest, and f₀(x−1, y), f₀(x+, y), f₀(x, y+1), and f₀(x, y−1)represent pixels removed by one pixel each in the x and y directions onthe space), and values obtained from the features within a predeterminedrange (sum, average, dynamic range, maximum value, minimum value,central value, or dispersion, etc.), and so forth, but may be others aswell.

The features altering unit 822 generates new features from the featuresstored in the features database 823, based on operating signals inputfrom the user. In further detail, the features altering unit 822 willgenerates new features information from features information stored inthe features database 823, based on operating signals input from theuser, and the features extracting unit 422 extracts new features basedon of this features information.

Also, saying that there is features information for features A, B, andC, as types of features, stored in the features database 823, anarrangement may be made wherein the DR (dynamic range: of valuegenerated by reading out the value for the feature A for multiple pixelsthat exist at a predetermined position as to the pixel of interest, andobtaining of value which is the difference between the minimum value andmaximum value, for each pixel) of the feature A may be made to befeature information corresponding to a new feature A′, or may be made tobe new features information by similarly obtaining sum, average, dynamicrange, maximum value, minimum value, central value, dispersion, thenumber of pixels having a value equal to or greater than a thresholdvalue (the threshold value may also be set), or a linear combinationbetween multiple features. Note that a linear combination betweenmultiple features is, for example, in the event that the Laplacian for apredetermined number of pixels is X_(a), Sobel is X_(b), and inter-framedifference is X_(c) with a linear combination of these three types offeatures being taken, the coefficients of these three types of featuresmultiplied and summed, i.e., A×x_(a)+B×x_(b)+C×x_(c) is the new featureof this pixel. Here, A, B, and C are coefficients, and in the case oftelop extraction for example, these coefficients can be obtained by thelearning processing when the pixels in the range that has been generallyspecified to for the telop portion by the operating unit 802 as thetutor data, and the multiple features as student data, as described withFIG. 43 and FIG. 44. In FIG. 81, with regard to the features stored inthe features database 823, features A through C indicate the types offeatures extracted by the features extracting unit 422, and features A′through C′ the types of features altered from the features A through Cby the features altering unit 822 (what is actually stored is and thefeatures information of each (the type of features, and informationindicating the detection method thereof)).

The features information stored in the features database 823 may bethose specified that the timing of being extracted by the featuresextracting unit 422 being stored, maybe those wherein featuresinformation stored in the features database 823 beforehand is alteredand by the features altering unit 822, or may be stored with othermethods.

Next, the telop extraction optimizing processing by the optimizingdevice 801 in FIG. 80 will be described with reference to the flowchartin FIG. 82. In step S381, the features extracting unit 422 of thefeatures detecting unit 411 judges whether or not two types of featureshave been selected by the features selecting unit 821, and in the eventthat these are not selected, the processing proceeds to step S386. Also,in the event that, for example, judgment is made that information forselecting features has been input from the features selecting unit 423,i.e., in the event that judgment is made that the user has operatedoperating unit 802 and selected two types of features, the processingproceeds to step S382.

In step S382, the features extracting unit 22 extracts the two types ofselected features from the image signals input signals for each pixel,and outputs to the processing deciding unit 412. At this time, signalsserving as the input signals are stored in the buffer 421.

In step S383, the processing deciding unit 412 decides the processingcontents for each pixel based on the input two types of features, andoutputs to the processing unit 413.

In step S384, the processing unit 413 processes the image signals whichare input signals input via the buffer 421 following the processingcontents input from the processing deciding unit 412, converts theseinto image signals that can be displayed on the display unit 403, andoutputs to the display unit 403 and displays.

In step S385, the features detecting unit 411 judges whether or not thetelop is deemed to be extracted. That is, the user views the imagedisplayed on the display unit 403, and in the event of not judging thatthe telop has been extracted, operates operating unit 802 so as tochange the combinations of features again and attempt telop extractionprocessing. In the event that operating signals corresponding to thisoperation are input from operating unit 802, the processing proceeds tostep S386.

In step S386, the features altering unit 822 of the features detectingunit 811 judge whether or not altering of the features has beenspecified, and in the event there is no instruction for alteration ofthe features, the processing returns to step S381. On the other hand, inthe event that the features altering unit 822 judges in step S386 thatthere has been operating signals input from the operating unit 802 whichinstruct alteration of the features, the processing proceeds to stepS387.

In step S387, the features altering unit 822 judges whether or not basicfeatures have been specified, and the processing is repeated untilinformation specifying the basic features is input. For example, in theevent that the operating unit 802 is operated and operating signalinstructing the feature A is input, judgment is made that a featurebased on the feature A in the image signals which is the input signalshas been input, and the processing proceeds to step S388.

In step S388, the features altering unit 822 judges whether or notalteration processing contents have been instructed, and repeats theprocessing until alteration processing contents are instructed. Forexample, in the event that the operating unit 802 is operated andoperating signals instructing DR are input, judgment is made that thealteration processing contents have been specified, and the processingproceeds to step S389.

In step S389, the features altering unit 822 alters the specifiedfeatures with the specified alteration processing contents to obtain newfeatures, which is stored in the features database 823, and theprocessing returns to step S381. That is to say, in this case, thefeatures altering unit 822 reads the features A out of the featuresdatabase 823 and obtains DR, which is the specified alterationprocessing contents thereof, thereby generating the new features A′,which is stored in the features database 823, and the processing returnsto step S381.

On the other hand, in the event that judgment is made in step S385according to the subjectivity of the user that a telop is beingextracted, operating signals indicating completion of the processing areinput to the features detecting unit 421 by the user operating theoperating unit 802, and the processing ends at this time.

That is to say, the processing of steps S381 through S389 is repeateduntil the user can judge that a telop is being extracted by viewing theimage displayed on the display unit 403, due to the above-describedprocessing, whereby a combinations of features optimal for the user canbe set, and telops can be extracted from image signals which are inputsignals, and further, even more combinations of features can be set byincreasing the types of features for the user to select, therebyexecuting processing that is optimal for the user.

In the above processing, processing contents are decided according tothe two types of features which the user has specified, and telop isextracted from the image signals which are input signals, so it can besaid that the “contents of processing” are changed such that outputsignals desired by the user are obtained, following operations of theuser. Also, the two axes of features (the two types of features beingselected) are switched over, while new features are also set (the typesof features increase), and that algorithm for deciding the processingcontents (e.g., whether or not to process as a telop) is changed by thecombination of the features, so it can be said that, of “contents ofprocessing”, the “structure of processing” is also changed.

Also, telop extraction is obtained by switching over the types offeatures to be extracted by trial and error, but with methods usingfeatures in a fixed manner, there has been the need to rewrite theprogram or remake the system itself in order to change the featuresanticipated, so the system has to be remake many times to obtain thealgorithms optimal for telop extraction by discovery, and has been quitedifficult in reality. Conversely, with the optimizing device accordingto the present invention, new features can be extracted in real-time,and the features distribution thereof can be shown, so the trial anderror performed by the user becomes easier, and the possibility offinding features optimal for telop extraction is improved.

Next, the configuration of an optimizing device 901 arranged byproviding the optimizing device 801 shown in FIG. 80 with the featuresdetecting unit 911 and internal information generating unit 912provided, will be described with reference to FIG. 83. The optimizingdevice 901 shown in FIG. 83 is basically the same as the optimizingdevice 801 shown at FIG. 80 and configuration, except that the featuresdetecting unit 911 is provided instead of the features detecting unit811, and further, an internal information generating unit 912 has beennewly provided.

Next, the configuration of the features detecting unit 911 will bedescribed with reference to FIG. 84. The features detecting unit 911 isbasically the same configuration as the features detecting unit 811 inFIG. 81, but is provided with the features selecting unit 921 instead ofthe features selecting unit 821, and a features altering unit 922instead of the features altering unit 822. The basic functions are thesame for each, but the features selecting unit 911 differs in thatinformation of that type of selected features is supplied to the ofinternal information generating unit 912. Also, features altering unit922 differs in that image information relating to instructions ofalteration contents are output to the display unit 403.

The internal information generating unit 912 is the same as the internalinformation generating unit 511 shown in FIG. 60. Also, the operatingunit 902 is the same as the operating unit 402.

Next, the telop extraction optimizing processing with the optimizingdevice 901 shown in FIG. 83 will be described, with reference to theflowchart in FIG. 85. Now, this processing is basically the same as theprocessing described with reference to the flowchart in FIG. 82, butdiffers in that the features selected by the internal informationgenerating unit 912 are displayed.

That is to say, in step S391, the features extracting unit 422 of thefeatures detecting unit 911 judges whether or not two types of featureshave been selected by the features selecting unit 921, and in the eventthat these have not been selected, the processing proceeds to step S397.Also, in the event that judgment has been made to that information forselecting the features have been input from the features selecting unit921, for example, the processing proceeds to step S392.

In step S392, the internal information generating unit 912 extracts theinformation indicating that types of the two selected types of featuresfrom the features selecting unit 921, and displays the names of thetypes of the two selected features on the display unit 403.

In step S393, the features extracting unit 422 extracts the two types ofselected features from that image signals which are the input signals,for each pixel, and outputs these to the processing deciding unit 412.At this time, the image signals which are the input signals are storedin the buffer 421.

In step S394, the processing deciding unit 412 decides the processingcontents for each pixel based on the two types of features that had beeninput, and outputs to the processing unit 413.

In step S395, the processing unit 413 processes the image signals whichare input signals input via the buffer 421 following the processingcontents input from the processing deciding unit 412, converts theseinto image signals that can be displayed on the display unit 403,outputs to the display unit 403, and displays.

In step S396, the features detecting unit 411 judges whether or not thetelop is deemed to have been extracted. That is, the user views theimage displayed on the display unit 403, and in the event of not judgingthat the telop has been extracted, operates the operating unit 902 so asto change the combinations of features again and attempt telopextraction processing. In the event that operating signals correspondingto this operation are input from operating unit 902, the processingproceeds to step S397.

In step S397, the features altering unit 922 of the features detectingunit 911 judges whether or not altering of the features has beenspecified, and in the event there is no instruction for alteration ofthe features, the processing returns to step S391. On the other hand, instep S397, in the event that the features altering unit 922 judges thatthere have been operating signals input from the operating unit 902which instruct alteration of the features, the processing proceeds tostep S398.

In step S398, the features altering unit 922 displays an instructionscreen for alteration contents, such as shown in FIG. 86, for example.In FIG. 86, the basic features display unit 931 is provided to the leftside in the figure, where features serving as a basis, which arefeatures currently stored in the features database 823, are displayed.In this case, features A through C and A′ through C′ are displayed. Alsodisplayed to the right side thereof is an alteration processing contentselection space 932. In this case, DR, maximum value, minimum value,central value, sum, dispersion, the number of pixels having a valueequal to or greater than a threshold value, and linear combination aredisplayed, with the space 932 a for setting the threshold value providedwith regard to the number of pixels having a value equal to or greaterthan a threshold value, and further, the space 932 b for selectingfeatures in the event of selecting the linear combination. Further, ascale settings space 933 is a displayed for settings scales of thevalues. A scale value indicates the range centered on the pixel ofinterest, and is of value indicating the range of pixels such as, forexample, in the case of detecting DR, setting the necessary pixels to 3pixels×3 pixels or 5 pixels×5 pixels, and so forth.

In step S399, the features altering unit 922 judges whether or notfeatures serving as a basis have been specified, and repeats theprocessing until information for specifying the features as a basis isinput. For example, in the event that the operating unit 902 is operatedand operating signals instructing the feature A are input, judgment ismade that the feature A has been input as a feature serving as a basisfor the image signals which are input signals, and the processingproceeds to step S400.

In step S400, the features altering unit 922 judges whether or notalteration processing contents have been instructed, and the processingis repeated until the alteration processing contents are instructed. Forexample, in the event that the operating unit 902 is operated andoperating signals instructing DR are input, judgment is made that thealteration processing contents have been instructed, and the processingproceeds to step S401.

In step S401, the features processing unit 922 alters the specifiedfeatures with the specified alteration processing contents, stores thisin the features database 823, and the processing returns to step S391.

On the other hand, in the event that judgment is made according to thesubjectivity of the user that a telop is being extracted, operatingsignals indicating completion of the processing are input to thefeatures detecting unit 421 by the user operating the operating unit902, and the processing ends at this time.

That is, due to the above-described processing, the user performsoperation while viewing the features serving as a basis and thealteration processing contents, and accordingly can recognize thefeatures whereby optimal processing can be performed. Further,subsequently, features capable of realizing optimal processing for theuser can be immediately specified. Also, the type of features which theuser specifies can be increased by inputting input information necessaryfor generating new features, following the display screen, socombinations of many features can be efficiently set.

In the above processing, the processing contents are decided accordingto two types of features that have been specified by the user, the telopis extracted from the image signals which are the input signals, so itcan be said that the “contents of processing” are being changed so thatoutput signals desired by the user can be obtained, following operationsmade by the user. Also, the two axes of the features (the two types offeatures that are selected) are switched over, while new features arealso set (the types of features increase), and the algorithm fordeciding the contents of processing (e.g., whether or not to process asa telop) is changed by the combination of these features, so of the“contents of processing”, the “structure of processing” is also changed.

Next, the configuration of an optimizing device 1001 which is providedwith an internal information generating unit 1011 instead of theinternal information generating unit 912 of the optimizing device 901shown in FIG. 83, will be described with reference to FIG. 87. In FIG.87, the structure is the same as that of the optimizing device 901 shownin FIG. 83, other than the internal information generating unit 912being replaced with the internal information generating unit 1011, andfurther, the processing deciding unit 412 being replaced with theprocessing deciding unit 711 shown in FIG. 69.

The internal information generating unit 1011 is basically the same asthat internal information generating unit 912, but further reads outinformation of processing contents decided for each pixel, decided bythe processing contents deciding unit 432 of the processing decidingunit 412, and displays this on the display unit 403 (the functions ofthe internal information generating unit 611 shown in FIG. 69 have beenadded).

Also, the operating unit 1002 is the same as the operating unit 402.

Next, the telop extraction optimizing processing with the optimizingdevice 1001 shown in FIG. 87 will be described with reference to theflowchart in FIG. 88.

In step S411, the buffer 421 of the features detecting unit 911 receivesimage signals which are input signals and stores these, and theprocessing unit 413 reads out the image signals which are input signalsstored in the buffer 421 without change, and output these to the displayunit 403 without any processing, where they are displayed.

In step S412, the processing contents deciding in its 721 of theprocessing deciding unit 711 judges whether are not the telop andbackground have been instructed by the operating unit 1002, theprocessing is repeated until the telop and background are instructed,and upon the telop and background being instructed, the processingproceeds to step S413.

In step S413, the features extracting unit 422 of the features detectingunit 911 judges whether or not two types of features have been selectedby the features selecting unit 921, and if not selected, the processingproceeds to step S421. Also, in the event that judgment is made thatinformation selecting the features has been input from the featureselecting unit 921, for example, the processing proceeds to step S414.

In step S414, the internal information generating unit 912 extractsinformation indicating the types of the two types of features that havebeen selected from the features selecting unit 921, and displays(presents) names of the types of the two types of features that havebeen selected, on the display unit 403.

In step S415, the features extracting unit 422 extracts the selected twotypes of features for each of the pixels from the image signals whichare input signals, and outputs these to the processing deciding unit412. At this time, the buffer 421 stores the image signals which areinput signals.

In step S416, the processing deciding unit 711 decides the processingcontents for each of the pixels based on the two types of features thathave been input, and output these to the processing unit 413.

In step S417, the internal information generating unit 1011 generates a2-dimensional distribution based on the two types of features input tothe processing deciding unit 711 and information of the pixel positionspecified for the telop and background, with the two types of featuresas the axes thereof, and in step S418, displays the 2-dimensionaldistribution on the display unit 403. In further detail, the featuresrecognizing unit 431 of the processing deciding unit 711 recognizes thetypes of features, and outputs the information indicating that the typethereof, and the features themselves to the processing contents decidingunit 721, the processing contents deciding unit 721 outputs thisinformation indicating the pixel positions specified for the telop inthe background, in addition to the information indicating the featuresand the types thereof, to the internal information generating unit 1011,whereby the internal information generating unit 1011 generates a2-dimensional distribution diagram of the features such as shown in FIG.74, for example, based on the information indicating the pixel positionsspecified for the telop and background.

In step S419, the processing unit 413 processes the image signals whichare input signals, input via the buffer 421, following the processingcontents input from the processing deciding unit 711, converts theseinto image signals which can be displayed on the display unit 403,outputs to the display unit 403, and displays.

In step S420, the features detecting unit 911 judges whether or not thetelop is deemed to have been extracted. That is, the user views theimage displayed on the display unit 403, and in the event of not judgingthat the telop has been extracted, operates the operating unit 1002 soas to change the combinations of features again and attempt telopextraction processing. In the event that operating signals correspondingto this operation are input from operating unit 1002, the processingproceeds to step S421.

In step S421, the features altering unit 922 of the features detectingunit 911 judges whether or not altering of the features has beenspecified, and in the event there is no instruction for alteration ofthe features, the processing returns to step S413. On the other hand, instep S421, in the event that the features altering unit 922 judges thatthere have been operating signals input from the operating unit 1002which instruct alteration of the features, the processing proceeds tostep S422.

In step S422, the features altering unit 922 displays an instructionscreen for alteration contents(FIG. 86).

In step S423, the features altering unit 922 judges whether or notfeatures serving as a basis have been specified, and the processing isrepeated until the features serving as a basis have been input. In theevent that judgment is made that features serving as a basis have beeninput, the processing proceeds to step S424.

In step S424, the features altering unit 922 judges whether or notalteration processing contents have been instructed, and the processingis repeated until the alteration processing contents are instructed. Inthe event that judgment is made that the alteration processing contentshave been input, the processing proceeds to step S425.

In step S425, the features altering unit 922 alters the specifiedfeatures serving as a basis with the specified alteration processingcontents, stores this in the features database 823, and the processingreturns to step S413.

On the other hand, in the event that judgment is made in step S420according to the subjectivity of the user that a telop is beingextracted, operating signals indicating completion of the processing areinput to the features detecting unit 421 by the user operating theoperating unit 1102, and the processing ends at this time.

That is, due to the above-described processing, the processing in stepsS411 through S425 is repeated until the user can judge from viewing thatimage displayed on the display unit 403 that the telop is extracted,thereby enabling a combination of features optimal for the user to beset and the telop to be extracted from the image signals which are inputsignals, and increasing the types of features which can be selected bythe user enables a great number of combinations of features to be set,so as to execute processing that is optimal for the user. Also, analteration instruction screen necessary for altering already-existingfeatures and generating new features is displayed, so the user canefficiently execute the alteration processing following the display.Also, at this time, the telop extraction processing can be repeatedlyexecuted while changing the features while viewing the state ofseparation between telop and background with regard to the features ofthe pixels, so features capable of highly precise telop extraction canbe easily selected.

Next, the configuration of an optimizing device 1101 provided with afeatures detecting unit 1111, processing deciding unit 1112, andoperating unit 1102, instead of the features detecting unit 411,processing deciding unit 412, and operating unit 402, of the optimizingdevice 401 shown in FIG. 55, will be described with reference to FIG.89. In FIG. 89, the configuration is the same as that of the optimizingdevice 401 shown in FIG. 55, other than the features detecting unit1111, processing deciding unit 1112, and operating unit 1102 being newlyprovided.

Of the configurations of the features detecting unit 411 in FIG. 56,features detecting unit 1111 is not provided with the features selectingunit 423, with the features extracting unit 422 extracts two types offeatures that have been set beforehand, but otherwise is of the sameconfiguration.

The processing deciding unit 1112 stores history information whereby theLUT stored in the processing contents database 433 is updated, andupdates the LUT corresponding to the history information. Theconfiguration of the processing deciding unit 1112 will be describedlater with reference to FIG. 90.

Also, the operating unit 1102 is the same as the operating unit 402.

Now, the configuration of the processing deciding unit 1112 will bedescribed with reference to FIG. 90. The processing deciding unit 1112in FIG. 90 is of the same configuration of the processing deciding unit412 other than being provided with the processing contents deciding unit1121 instead of the processing contents deciding unit 432 of theprocessing deciding unit 412 shown at FIG. 57, and further, havinghistory memory 1122 added thereto.

The processing contents deciding unit 1121 stores history information ofthe operations for changing the LUT stored in the processing contentsdatabase 433, in the history memory 1122, and changes the LUT based onthe history information. Other functions are also the same as those ofthe processing contents deciding unit 432 of the processing decidingunit 412 shown in FIG. 57, as well.

Next, the telop extraction optimizing processing by the optimizingdevice 1101 shown in FIG. 89 will be described with reference to theflowchart shown in FIG. 91.

In step S431, the features extracting unit 411 of the features detectingunit 1111 extracts two predetermined types of features from the imagesignals which are input signals, and outputs these to the processingdeciding unit 1112. At this time, the buffer 421 stores the imagesignals which are input signals.

In step S432, the processing contents deciding unit 1121 of theprocessing deciding unit 1112 decides the processing contents for eachpixel making reference to the LUT stored in the processing contentsdatabase 433, based on the types of features and the features input fromthe features recognizing unit 431, and outputs these to the processingunit 413.

In step S433, the processing unit 413 processes the pixels following theprocessing contents input from the processing deciding unit 1112,outputs to the display unit 403, and displays.

In step S434, the processing contents deciding unit 1121 of theprocessing deciding unit 1112 to this whether or not operating signalsfor changing the LUT have been input from the operating unit 1102. Thatis, the user views the image displayed on the display unit 403 andsubjectively judges whether processing which is according to thepreferences of the user is being performed, the user operates theoperating unit 1102 based on the judgement results, so thatcorresponding operating signals are input. For example, in the eventthat processing matching the preferences of the user is not beingperformed (in the event that a screen according to the preferences ofthe user is not being displayed on the display unit 403), a request forchanging the LUT is input for changing the LUT in the processingcontents database 433 of the processing deciding unit 1121.

In step S434, in the event that operating signals requesting change ofthe LUT have been input, i.e., processing according to the preferencesof the user is not being performed, the processing proceeds to stepS435.

In step S435, judgment is made regarding whether or not the changingprocessing of the LUT can be executed by auto LUT changing processing.Here, manual LUT changing processing and auto LUT changing processingexist for changing processing for the LUT. The details of that judgmentregarding whether or not auto LUT changing processing is capable will bedescribed later.

For example, in the event that due to and is made in step S435 that autoLUT changing processing is not possible, the processing proceeds to stepS436, where manual LUT changing processing is executed.

Now, along with a description of the LUT changing processing, thedetails of the LUT will be described. The LUT is a table indicatingprocessing contents which are decided for each of the combinations oftwo features, as shown in FIG. 92. In FIG. 92, a case of features A andB as the two types of features is illustrated, and in this example, anexample is shown wherein the features are classified into 8 steps (atotal of 64 classes). Also, in FIG. 92, the features are normalized tovalues of 0 through 1.0, and are divided into the ranges of, with regardto the values V_(a) of the feature A, from the left, 0≦V_(a)<⅛,⅛≦V_(a)<{fraction (2/8)}, {fraction (2/8)}≦V_(a)<⅜, ⅜≦V_(a)<{fraction(4/8)}, {fraction (4/8)}≦V_(a)<⅝, ⅝≦V_(a)<{fraction (6/8)}, {fraction(6/8)}≦V_(a)<⅞, ⅞≦V_(a)≦{fraction (8/8)}, and, with regard to the valuesV_(b) of the feature B, from the top, 0≦V_(b)<⅛, ⅛≦V_(b)<{fraction(2/8)}, {fraction (2/8)}≦V_(b)<⅜, ⅜≦V_(b)<{fraction (4/8)}, {fraction(4/8)}≦V_(b)<⅝, ⅝≦V_(b)<{fraction (6/8)}, {fraction (6/8)}≦V_(b)<⅞,⅞≦V_(b)≦{fraction (8/8)}. In the figure, the processing contents areclassified into the three types of X, Y, and Z, according to thecombinations of features of the ranges, and in the case shown in FIG.92, in the range of 0≦V_(a)≦⅜ and 0≦V_(b)≦⅜, the processing contents areX, in the range of {fraction (4/8)}≦V_(a)<{fraction (6/8)} or {fraction(4/8)}≦V_(a)<{fraction (6/8)}, the processing contents are Y, and inother ranges, the processing contents are Z. Also, variousspecifications can be made for the processing contents. For example, asshown in FIG. 93 through FIG. 95, prediction taps used for processing inthe processing unit 413 can be specified with regard to pixels ofinterest.

That is, with FIG. 93, the processing contents X are illustrated, withthe pixel of interest as P0, wherein filter taps P1 and P2 are set withthe pixel of interest P0 in the center in the x direction spatially, andin the same manner filter taps P3 and P4 are set with the pixel ofinterest P0 in the center in the y direction, and further, filter tapsP5 and P6 are set before and after with the pixel of interest P0 in thecenter time-wise (e.g., tap P6 at the same pixel positions and in oneframe previous and P5 in one frame later). That is to say, theprocessing contents X are at a so-called space-time filteringprocessing.

Also, in FIG. 94, the processing contents Y are illustrated, wherein,instead of the taps P3 and P4 in the time-space filtering in FIG. 93, atap P12 of the timing further previous in the time-wise direction thanthe tap P6, and a tap P11 of the timing further later than the tap P5,are set here, that is, processing contents Y are so-called timefiltering processing.

Further, in FIG. 95, the processing contents Z are illustrated, wherein,instead of the taps P5 and P6 in the time-space filtering in FIG. 93, atap P21 distanced from the pixel of interest farther than the tap P1 inthe x direction, and a tap P22 distanced from the pixel of interestfarther than the tap P2, are set. That is, the processing contents Z areso-called space filtering processing.

Note that the processing contents are not restricted to the three typesas with a an example shown in FIG. 92 with regard to the types thereof,and as a matter of course may be divided into a number of types otherthan this number, for example, an arrangement may be made wherein allpixels are subjected to binarization processing for dividing into whiteor black. Also, this binarization processing may be, as described in theexample above, for example, that which specifies binarization processingfor whether or not a pixel is to be extracted as a telop portion.Conversely, there may be three types or more for the types of processingcontents.

Next, the manual LUT changing processing in step S436 in FIG. 91 will bedescribed with reference to the flowchart in FIG. 96.

In step S441, processing contents deciding unit 1121 judges whether ornot a pixel position is specified in processing contents are specifiedas operating signals from the operating unit 1102, and the processing isrepeated until a pixel position is specified and processing contents arespecified as operating signals from the operating unit 1102. That is tosay, for example, in the event that the screen shown in FIG. 97 isdisplayed on the display unit 403, the user can operate the pointer 1131on the image displayed on the display unit 403 to execute predeterminedprocessing at the pixel position to which the users desires to add achange in processing, displaying a drop-down list 1132 as shown in FIG.97, for example, and further specifying one of the processing contentsX, Y, and Z displayed in the drop-down list 1132. In step S441, uponthis specification being made, judgment is made that the pixel positionis specified and the processing contents are specified, and theprocessing proceeds to step S442. In this case, as shown in FIG. 97, thepixel position P41 is selected, and processing contents X are selected.

In step S442, the processing contents deciding unit 1121 reads out thecorresponding combination of two types of features from the specifiedpixel position. In further detail, of the features detected by thefeatures detecting unit 1111, the processing contents deciding unit 1121reads out the combination of the two types of features corresponding tothe specified pixel.

In step S443, the processing contents deciding unit 1121 updates theprocessing contents matching the corresponding combination of features,to the processing contents specified in step S441.

In step S444, the processing contents deciding unit 1121 stores thechanged pixel position and the processing contents in the history memory1122.

In step S445, the processing contents deciding unit 1121 judges whetheror not there are other changes to the LUT, and in the event thatjudgment is made that there are yet changes to be processed for the LUT,i.e., in the event that operations signals instructing other changes tothe LUT are input from the operating unit 1102, the processing returnsto step S441, and in the event that judgment is made there is noprocessing to continue changing the LUT, i.e., in the event thatoperating signals indicating ending of the changing of the LUT are inputfrom the operating unit 1102, the processing ends.

With the above manual LUT changing processing, changing of the LUT isperformed as follows. That is, in step S442, in the event that, forexample, (V_(a), V_(b))=(0.52, 0.27), for the features of the pixel atthe obtained pixel position P41, the processing contents Y are set inFIG. 98 for the position (position from the left, position from thetop)=(5, 3)on the LUT in FIG. 98 (positions on the LUT will be indicatedin the same manner hereafter, as well). In the event that the processingcontents are changed to X in step S443 as shown in FIG. 97, theprocessing at the position (5, 3) on the LUT is changed from theprocessing contents Y to the processing context X in step S443, as shownin FIG. 99.

After the manual LUT changing processing shown in FIG. 96 is ended, theprocessing returns to the processing in step FIG. 432 in the flowchartin FIG. 91, the subsequent processing is repeated. In this case, in theevent that the LUT is a changed as described above, processingcorresponding to the processing contents Y is executed up to the pixelof the LUT wherein the combination of the two types of features is (5,3), as shown in FIG. 99. Consequently, of the image shown in FIG. 97,processing is changed corresponding to the pixel having the samefeatures as the pixel position P41, i.e., the pixel belonging to (5, 3)on the LUT, and images displayed wherein processing different from thatin the case of FIG. 97 is performed on that portion, as shown in FIG.100, for example. Note that in FIG. 100, an example is shown wherein apixel having the same features as the pixel at pixel position P41 inFIG. 97 is white.

While description has been made regarding an example wherein (processingcontents on) a LUT specified by a combination of teachers for a selectedpixel alone are changed, with the processing example described withreference to the flowchart in FIG. 96, changes to the LUT are notrestricted to this. That is, for example, an arrangement may be madewherein all nearby portions from the portion where change to the LUT hasbeen specified on the LUT, for example, are changed processing contents.That is, for example, as shown in FIG. 101, in the event that allprocessing contents are set as Z for the initial state of the LUT (theinitial state of the LUT is a so-called default setting, and may be astate wherein all processing contents are not Z), an arrangement may bemade wherein, in the event that the processing contents of the position(4, 2) on the LUT are changed to X and the processing contents at theposition (5, 2) are changed to Y, as shown in FIG. 102A, due to theprocessing of the flowchart in FIG. 96, areas near to the changedpositions on the LUT may be changed to X or Y as shown in FIG. 102B.That is, in the case of FIG. 102B, the distance from the position (4, 2)on the LUT and the distance from the position (5, 2) thereupon iscompared for all positions on the LUT, with all being changed to theprocessing contents of the position which is close. Consequently, inFIG. 102B, the left half region is made to be the processing contents X,while the right half region is made to be the processing contents Y.

In the same way, as shown in FIG. 103A, in the event that the processingcontents of the combination the features at the positions (4, 2), (7, 7)on the LUT are changed to X, and the processing contents of thecombination the features at the positions (5, 2), (4, 5) on the LUT arechanged to Y, the processing contents at (1, 1), (2, 1), (3, 1), (4, 1),(1, 2), (2, 2), (3, 2), (4, 2), (1, 3), (2, 3), (3, 3), (4, 3), (5, 7),(5, 8), (6, 6), (6, 7), (6, 8), (7, 5), (7, 6), (7, 7), (7, 8), (8, 4),(8, 5),)8, 6), (8, 7), (8, 8), on the LUT are changed to X, as shown inFIG. 103B, and processing contents are Y at all other positions on theLUT.

Further, in the same manner, as shown in FIG. 104A, in the event thatthe processing contents of the combination the features at the positions(4, 2), (7, 7) on the LUT are changed to X, and the processing contentsof the combination the features at the positions (2, 3), (5, 2), (4, 5),(7, 4) on the LUT are changed to Y, the processing contents at (3, 1),(3, 2), (4, 1), (4, 2), (4, 3), (5, 7), (5, 8), (6, 6), (6, 7), (6, 8),(7, 6), (7, 7), (7, 8), (8, 6), (8, 7), (8, 8), on the LUT are changedto X, as shown in FIG. 104B, and processing contents are Y at all otherpositions on the LUT.

Changing the LUT thus enables batch changing of the processing contentsof a combination of multiple features at a single time of processing,with a relatively approximated combination of features.

Now, let us return to description of the flowchart in FIG. 91.

In step S435, and the event that judgment is made that auto LUT changingprocessing as possible, the flow proceeds to step S437, and auto LUTchanging processing is executed.

Now, the auto LUT changing processing will be described, with referenceto the flowchart in FIG. 105. In step S461, the processing contentsdeciding unit 1121 obtains a group which exists for each of theprocessing contents in the distribution of updating history stored inhistory memory 1122. That is to say, the history memory 1122 stores ahistory table which is different from the LUT, recording, for example,the position on the LUT regarding which changing has been specified bythe above-described manual LUT changing processing, and the specifiedprocessing contents, as shown in FIG. 106. The history table shown inFIG. 106 indicates that change has been instructed for the processingcontents for (3, 3), (8, 3), (2, 5), (6, 6) on the LUT, with theprocessing contents X, X, X, and Y being instructed, respectively.

Now, a group for each processing contents means an area wherein theprocessing contents on the history table exist with a predetermineddensity or higher, over a predetermined area or greater.

For example, in the event that a group exists, a group 1151 is formed onthe table as shown in FIG. 107, and the processing contents decidingunit 1121 *tames the group 1151. Note that in FIG. 107, the group 1151is a group of processing contents X, and groups are obtained for each ofother processing contents as well.

Also, a group does not exist unless manual LUT processing has beenexecuted a certain number of times, with the updating history beingstored in a history member 1122, and also with changing processing beingexecuted to the same processing contents at a position somewhat groupedtogether on the LUT. Now, in step S435 in FIG. 91, judgment is maderegarding whether or not auto LUT changing processing is possible, bywhether or not groups of the processing contents exist. That is, in theevent that a group exist for each other processing contents, judgment ismade that auto LUT changing processing is possible, and if the groups donot exist, judgment is made that auto LUT changing processing is notpossible.

In step S462, the processing contents deciding unit 1121 detects theposition of the center of gravity of a group obtained in step S461.

That is, for example, in the case in FIG. 107, the group 1151 is formed,and position of center of gravity thereof is obtained from all positionson the history table where processing contents X are specified. In thecase in FIG. 107, the center of gravity 1161 is obtained at allpositions on the history table where processing contents X are specifiedwithin the group 1161.

In step S463, the processing contents deciding unit 1121 converts theprocessing contents of grids on the LUT corresponding to the combinationof features of the respective pixels into processing contents of gridson the history table corresponding to the combination of features ofpixels making up groups, corresponding to the contents of processing atthe position on the history table existing within a predetermined rangefrom the position of center of gravity of the group, and the processingends. That is to say, in FIG. 107, all the processing contents on theLUT corresponding to the position on the history table exist in with inthe range 1162 which is the range of a circle having a predeterminedradius center to on the position of center of gravity 1161, are changedinto processing contents X making up the group.

That is, for example, as shown in FIG. 107, in the event that thehistory table is generated in the state with an LUT configured such asshown in FIG. 108, (2, 3), (3, 4) on the history table with in the range1162 within a predetermined distance from the center of gravity 1161store history information of processing contents Y which are a differentto the processing contents X making up the group, as shown in FIG. 107,so the processing contents of (2, 3), (3, 4) on the LUT corresponding to(2, 3), (3, 4) on the history table are changed to X. Accordingly, inthe event that LUT is configured as shown in FIG. 108, the processingcontents of (2, 3) on the LUT are Y, so the processing contents decidingunit 1121 changes the processing contents of (2, 3) on the LUT to X, asshown in FIG. 109. Also, in the case in FIG. 108, the processingcontents of (3, 4) on the LUT are X, so the processing contents decidingunit 1121 maintains this state.

Due to this processing, the processing contents on the LUT (gridinformation) is automatically changed. Note that this processing is notrestricted to being performed at the timing of the user instructingchanging processing of the LUT, and may be repeatedly executed atpredetermined time intervals.

Now, let us return to the description of the flowchart in FIG. 91.

In the event that judgment is made in step S434 is that change of theLUT is not instructed, i.e., in the event that the user its use theimage display of the display unit 403 and judges that an image suitingthe user's preferences is generated, the processing ends.

Repeating changing of the LUT following user operations by the aboveprocessing changes the processing contents registered to each of thegrids on the LUT, thereby enabling images suiting the preferences of theuser to be generated.

In the above, the processing contents on LUT are changed according touser operations, so it can be said that the “contents of processing” arechanged. Also, change is made into one or the other of: a processingsystem wherein the processing contents of that LUT are changed from theposition of center of gravity of the group; or a processing systemwherein, according to processing contents at a specified position on theLUT, the processing contents on the LUT at another position on the LUTare changed as shown in FIG. 101 through FIG. 104; depending on whetheror not the group is obtained for each processing content, according toprocessing performed by the user. That is, the changing algorithm forthe LUT is changed, so the “structure of processing” of the “contents ofprocessing” can also be said to be changed according to user operations.

Next, another configuration example of a optimizing device 1101 with theprocessing deciding unit 1181 provided instead of the processingdeciding unit 1112 of that optimizing device 1101 shown in FIG. 89, andan internal information generating unit 1182 newly provided, will bedescribed with reference to FIG. 110. In FIG. 110, the configuration isthe same as that of the optimizing device 1101 shown in FIG. 89 otherthan the processing deciding unit 1181 and internal informationgenerating unit 1182 being newly provided.

The processing deciding unit 1181 stores history information whereby theLUT stored in the processing contents database 1191 (FIG. 111) isupdated, and changes the LUT corresponding to the history information,while supplying the LUT stored in the processing contents database 1191to the internal information generating unit 1182.

The internal information generating unit 1182 reads out the LUT storedin the processing contents database 1191, converts this into informationwhich can be displayed on the display unit 403, outputs to the displayunit 403, and displays.

Next, the configuration of the processing deciding unit 1181 will bedescribed with reference to FIG. 111. The processing deciding unit 1181in FIG. 111 is of the same configuration as the processing deciding unit1112 except that the processing contents database 1191 is providedinstead of the processing contents database 433 shown in FIG. 90.

The processing contents database 1191 stores the LUT, and also suppliesinformation of the LUT to the Internal information generating unit 1182low as necessary. The other functions are the same as those of theprocessing contents database 433 shown in FIG. 90.

Next, the telop extraction optimizing processing by the optimizingdevice 1101 shown in FIG. 110 will be described, with reference to theflowchart shown in FIG. 112. Now, the telop extraction optimizingprocessing in FIG. 112 is basically the same as the processing describedwith reference to the flowchart in FIG. 91, wherein the processing insteps S471 through S473 and S475 through S478 in FIG. 112 is processingcorresponding to the steps S431 through S437 in FIG. 91. Following theprocessing in step S473, in step S474, the internal informationgenerating unit 1182 reads out the LUT of the processing contentsdatabase 1191 of the processing deciding unit 1181, converts this intoimage signals that can be displayed on the display unit 403, outputs tothe display unit 403, displays (presents) it, proceeds to step S475, andthe subsequent processing is repeated.

Due to such processing, the LUT is displayed (presented), so that LUTcan be changed while recognizing the processing performed on the imagesignals which are input signals, and change of the LUT, from the imagedisplayed on the display unit 403.

So far, an example has been described wherein, in manual LUT changingprocessing, the LUT is changed by specifying pixels on the imageprocessed with the processing contents registered in the LUT displayedon the display unit 403, and processing contents thereof, but anarrangement may be made wherein the internal information generating unit1182 reads out the LUT stored in the processing contents database 1191and, for example, displays the processing contents on the LUT on thedisplay unit 403 in a state that this directly operable by the operatingunit 1102, so that the processing contents on the LUT can be directlychanged.

Now, manual LUT changing processing for the optimizing device 1101 inFIG. 110 to directly change the values of the above-described LUT willbe described with reference to the flowchart in FIG. 113.

In step S481, the processing contents deciding unit 1121 decides whetheror not the position has been specified on the LUT, and the processing isrepeated until specified. For example, as shown in FIG. 114, in theevent that the position (5, 3) on the LUT displayed on the display unit403 as a position where the processing contents are set to Y isspecified, judgment is made that a position has been specified on theLUT, and the processing proceeds to step S482.

In step S482, the internal information generating unit 1182 makes adisplay on the display unit 403 indicating to position and on the LUTthat has been specified. That is, in the case in FIG. 114, positioneddisplayed frame 1192 is displayed at the specified (5, 3).

In step S483, the processing contents deciding unit 1121 decides whetheror not the processing contents have been specified, and repeats theprocessing until the processing contents are specified. For example, asshown in FIG. 114, specifying by a displaying a drop-down list 1193 atthe position where the pointer 1191 is operated (e.g., by right-clickingthe mouse serving as the operating unit 1102), and selecting one of theprocessing contents X, Y, and Z, by the user operating the operatingunit 1102, results and judgment that processing contents have beenspecified, and the processing proceeds to step S484.

In step S484, the processing contents deciding unit 1121 changes theprocessing contents to the specified processing contents, and ends theprocessing. That is, in the case of FIG. 114, “X” displayed in thedrop-down list 1193 is selected, so, and as shown in FIG. 115, theprocessing contents of (5, 3) on the LUT are changed from Y to X.

Due to the above processing, the processing contents set on the LUT canbe directed changed, and LUT can be operated a while viewing and imageprocessed by the processing contents registered to the LUT, soprocessing contents suiting the preferences of the user can be readilyset.

With optimizing device 1101 in FIG. 110, the processing contents on thegrids on the LUT specified by user operations are changed in the manualLUT changing processing, so it can be said that the “contents ofprocessing” are changed by user operations. Also, upon changing historystored in history memory 1122 being accumulated to a certain degree, anda group being detected, the algorithms for changing the LUT are changedfrom manual LUT changing processing to auto LUT changing processing, sothe “structure of processing” is being changed.

Further, the LUT is displayed is internal information relating toprocessing of the processing deciding unit 1112, while the processingcontents of the LUT can be changed while viewing the display LUT, so theuser can recognize the correlation relationship between the contents onthe LUT and the image displayed on the display unit 403.

Next, the configuration of an optimizing device 1201 as anotherembodiment wherein the features detecting unit 1111 and processing unit413 are provided instead of the features detecting unit 411 andprocessing unit 413 of the optimizing device 401 shown in FIG. 55, willbe described, with reference to FIG. 116.

The features detecting unit 1111 is the same as that in theconfiguration of the optimizing device 1101 in FIG. 89.

The processing unit 1211 subjects input signals read out from the buffer421 to mapping processing using a coefficient set obtained by learning,for example, based on information of processing contents input from theprocessing deciding unit 413, outputs to the display unit 403, anddisplays. The processing deciding unit 413 changes the method of thelearning coefficient set, based on operating signals from the operatingunit 1202. Now, operating unit 1202 is the same as the operating unit402.

Next, the configuration of the processing unit 1211 will be describedwith reference to FIG. 117.

The learning device 1221 learns, by least-N'th-power error, coefficientsets necessary for mapping processing of the mapping processing unit1222 for each of the processing contents, based on the image signalswhich are input signals read out from the buffer 421 of the featuresdetecting unit 1111, and stores in the coefficient memory 1237. Also,the learning device 1221 learns coefficient sets by changing the valueof the exponent N in least-N'th-power error, based on operating signalsinput from the operating unit 1202.

The mapping processing unit 1222 reads out corresponding coefficientsets to on the coefficient memory 1237 of the learning device 1221,based on the processing contents input from the processing deciding unit412, performs a mapping processing of the image signals which are inputsignals read out from the buffer 421 of the features detecting unit1111, outputs to the display unit 403, and displays.

Next, the detailed configuration of the learning device 1221 will bedescribed with reference to FIG. 118. The tutor data generating unit1231 is the same as the tutor data generating unit 231 shown in FIG. 30,which generates tutor data from input signal serving as learning data,and outputs this to the least-N'th-power error coefficient computingunit 1236. The student data generating unit 1232 is the same as thestudent data generating unit 232 in FIG. 30, which generates tutor datafrom the input signals serving as learning data, and outputs this to thefeatures extracting unit 1233 and prediction tap extracting unit 1235.

The features extracting unit 1233 is the same as the features extractingunit 422 of the features detecting unit 1111, which extracts featuresfrom the student data and outputs these to the processing deciding unit1234. The processing deciding unit 1234 is the same as the processingdeciding unit 412, which decides processing contents based on thefeatures input from the features detecting unit 1233, and outputs thisto the least-N'th-power error coefficient computing unit 1236. Theprediction tap extracting unit 1235 is the same as a prediction tapextracting unit 233, which, with the tutor data sequentially as pixelsof interest, extracts from the student data, pixels serving asprediction taps with regard to the pixels of interest, and outputs tothe least-N'th-power error coefficient computing unit 1236.

The least-N'th-power error coefficient computing unit 1236 is the sameas the least-N'th-power error coefficient computing unit 234 shown inFIG. 30, with regard to the basic configuration and processing contents,which computes coefficient sets by least-N'th-power error from theprediction taps input from the prediction tap extracting unit 1235 andthe tutor data, based on information specifying the exponent N necessaryfor the least-N'th-power error coefficient computing processing inputfrom the operating unit 1202, and outputs this to the coefficient memory1237, where it is overwritten. However, the least-N'th-power errorcoefficient computing unit 1236 differs from the least-N'th-power errorcoefficient computing unit 234 in FIG. 30 in that coefficient sets aregenerated for each of the processing contents input from the processingdeciding unit 1234. The coefficient memory 1237 stores, for each of theprocessing contents, the coefficient sets being output from theleast-N'th-power error coefficient computing unit 1236 for each of theprocessing contents. FIG. 118 illustrates coefficient sets A through Nbeing stored for each of the processing contents.

Next, the configuration of the mapping processing unit 1222 will bedescribed with reference to FIG. 119. The tap extracting unit 251 is thesame as the mapping processing unit 222 in FIG. 31, which extractsprediction taps from input signal supplied from the buffer 421, withregard to the pixel of interest, and outputs to the sum of productcomputing unit 1251. The sum of product computing unit 1251 is the sameas the sum of product computing unit 252 shown in FIG. 31, which, uponinput from the prediction tap extracting unit 251, executes sum ofproduct computing processing using the value of the extracted predictiontaps (pixels) and the coefficient sets stored in the coefficient memory1237 of the learning unit 1221, generates the pixel of interest,performs this on all pixels and outputs to the display unit 403 asoutput signals, and displays. Note however, that in the sum of productcomputing processing, the sum of product computing unit 1251 uses, ofthe coefficient sets A through N stored in the coefficient memory 1237,the coefficient sets corresponding to the processing contents suppliedfrom the processing deciding unit 412.

Next, learning processing with image optimizing processing by theoptimizing device 1201 shown in FIG. 116, will be described withreference to the flowchart in FIG. 120.

In step S501, judgment is made regarding whether or not the user hasoperated the operating unit 202, and in the event that judgment is madethat this has not been operated, the flow returns to step S501. Also, inthe event that in step S501, judgment is made that the operating unit1202 has been operated, the flow proceeds to step S502.

In step S502, the tutor data generating unit 1231 of the learning device1221 generates tutor data from the input signals and outputs this to theleast-N'th-power error coefficient computing unit 1236, while thestudent data generating unit 1232 and generates student data from theinput signals and outputs this to the features extracting unit 1233 andprediction tap extracting unit 1235, and the flow proceeds to step S503.

Now, as for the data used for generating the student data and tutor data(hereafter may be referred to as learning data) input signals input at apoint in the past by a predetermined amount of time from the currentpoint, for example, may be employed. Also, for the learning data,dedicated data may be stored beforehand, instead of using input signals.

In step S503, the features extracting unit 1233 extracts features fromthe student data at a position corresponding to the pixel of interest(tutor data), and outputs to the processing deciding unit 1234.

In step S504, the processing deciding unit 1234 decides the processingcontents for the pixel of the interest based on the features input fromthe features extracting unit 1233, and outputs this to theleast-N'th-power error coefficient computing unit 1236. For example, theprocessing deciding unit 1234 may perform vector quantizing of one ormultiple features from the features extracting unit 1233, and use thequantizing results thereof as information of the processing contents.However, in this case, there is no LUT or the like stored as with theprocessing deciding unit 1112 in FIG. 89.

In step S505, the prediction tap extracting unit 1235 takes each pieceof tutor data as a pixel of interest, generates prediction taps from thestudent data input from the student data generating unit 1232 withregard to each of the pixel of interest, and outputs to theleast-N'th-power error coefficient computing unit 1234, and the flowproceeds to step S506.

In step S506, the least-N'th-power error coefficient computing unit 1236judges whether or not operating signals for specifying a computing of acoefficient set with least-N'th-power error by the recursive method(second method) have been input from the operating unit 202, and in theevent that judgment is made that the user has operated the operatingunit 1202, for example, to specify not the recursive method, i.e., thedirect method (first method), the flow proceeds to step S507, judgmentis made regarding whether coefficients a, b, and c, specifying theweight α_(s) (specifying the exponent N) in Expression (50) has beeninput, the processing is repeated until input, and in the event thatjudgment is made that, for example, the user has operated the operatingunit 1202 and values specifying the coefficients a, b, and c have beeninput, the processing proceeds to step S507.

Is step S507, the least-N'th-power error coefficient computing unit 1236solves the problem of minimizing the above-described Expression (48) isessentially with least-square-power error, in the state of thecoefficients a, b, and c, with the weight α_(s) input, thereby obtainingthe prediction coefficients w₁, w₂, w₃, . . . , w_(m), i.e., acoefficient set which is the solution for the exponent N correspondingto the weight α_(s), by least-N'th-power error, for each of theprocessing contents input from the processing deciding unit 1234, storesthese in the coefficient memory 1237, and the flow returns to step S501.

On the other hand, in the event that a judgment is made that therecursive method is selected in step S506, the processing proceeds tostep S509.

In step S509, the least-N'th-power error coefficient computing unit 1236judges whether or not information specifying the exponent N has beeninput, the processing is repeated until the exponent N is input, and inthe event that judgment is made to that, for example, the user has inputinformation's specifying the exponent N by operating the operating unit1202, the processing proceeds to step S510.

In step S510, the least-N'th-power error coefficient computing unit 1236obtains a coefficient set by a solution with the least-N'th-power errormethod serving as a basis. In step S511, the least-square errorcoefficient computing unit 1236 obtains the coefficient set byleast-N'th-power error corresponding to the input from the operatingunit 1202, in a recursive matter for the exponent N as described withreference to Expression (51) through Expression (54), using predictionvalues obtained from the coefficient sets obtained byleast-square-error, for each of the processing contents input from theprocessing deciding unit 1234, stores these in the coefficient memory1237, and the flow returns to step S501.

Due to the above processing, coefficient sets are learned in thecoefficient memory 1237, for each of the processing contents.

Next, the mapping processing in the image optimizing processing with theoptimizing device 1201 shown in FIG. 116 will be described, withreference to the flowchart in FIG. 121.

In step S521, the features detecting unit 1111 detects features ofpixels of the input signals at positions corresponding to the pixels ofinterest (output signals) from the image signals which are inputsignals, and outputs the detected features to the processing decidingunit 412.

In step S522, the processing deciding unit 412 decides the processingcontents based on the features input from the features detecting unit1111, and outputs this to the processing unit 1211. The processing ofthe processing deciding unit 412 decides the processing contents byperforming the same processing as the processing deciding unit 1234 inFIG. 118. Accordingly, as described above, in the event that theprocessing deciding unit 1234 performs vector quantizing on one ormultiple features from the features extracting unit 1233 and takes thequantizing results thereof as information for the processing contents,at described above, there is no LUT or the like stored in the processingunit 412 shown in FIG. 116, as with the processing deciding unit 1112 inFIG. 89.

In step S523, with the frame of an image serving as output signalscorresponding to an image frame serving as current input signals, as aframe of interest, the tap extracting unit 251 of the mapping processingunit 1222 of the processing unit 1211 takes, of the pixels of that frameof the interest, one which has not yet been made the pixel of interest,in raster scan order, for example, as the pixel of interest, extracts aprediction tap from input signals with regard to the pixel of interest,and outputs this to the sum of products computing unit 1251.

In step S524, the sum of products computing unit 1251 of the mappingprocessing unit 1222 will reads out coefficient sets corresponding tothe processing contents input from the processing deciding unit 412,from the coefficient memory 1237 of the learning device 1221.

Then, in step S525, the sum of products computing unit 1251 uses theprediction coefficient corresponding to the processing contents, readout from the coefficient memory 1237 of the learning device 1221, andexecutes sum of product computing processing with the prediction tapinput from the tap extracting unit 251 and a coefficient set read outfrom the coefficient memory 1237, following the Expression (39). Thus,the sum of products computing unit 1251 obtains the pixel value of thepixel of interest (prediction value). Subsequently, the flow proceeds tostep S526, where the tap extracting unit 251 judges whether or not allpixels in the frame of interest had been made to be the pixel ofinterest, and in the event that judgment is made otherwise, the flowreturns to step S521, a pixel which has not yet been made the pixel ofinterest is taken as a new pixel of interest in raster scan order of theframe of interest, and subsequently, the same processing is repeated.

Also, in the event that judgment is made in step S526 that all pixels inthe frame of interest have been made to be the pixel of interest, theflow proceeds to step S527, where the display unit 403 displays theframe of interest made from the pixels obtained with the sum of productcomputing unit 1251.

The flow then returns to step S521, or the features detecting unit 1111detects the features of pixels of input signals at a positioncorresponding to pixel the of the interest (output signals) from theimage signals which are input signals, the next frame is taken as thenew frame of interest, and subsequently, the same processing isrepeated.

Note that with the optimizing device 1201 in FIG. 116, the exponent N ischanged by the user operating the operating unit 1202 (in the directmethod, the coefficients a, b, and c specifying the exponent N arechanged, while in the recursive method, the exponent N itself ischanged), thereby setting what sort of exponent N is to be employed forthe least-N'th-power error method, as a learning norm (learning system)of prediction coefficients (coefficient set). That is, the learningalgorithms for obtaining coefficients are changed. Accordingly, it canbe said that the “structure of processing” is changed so as to obtain animage suiting the preferences of the user.

Next, the configuration of the optimizing device 1301 wherein theinternal information generating unit 1312 is provided to the optimizingdevice 1201 shown in FIG. 116, will be described with reference to FIG.122.

The optimizing device 3101 in FIG. 122 is the same as the optimizingdevice 1201 in FIG. 116, other than the processing unit 1311 is providedinstead of the processing unit 1211, by the internal informationgenerating unit 1312 being provided.

The internal information generating unit 1312 reads out coefficient setinformation stored for each of the processing contents from thecoefficient memory 1321 in the processing unit 1311, for example, asinternal information, converts this into information that can bedisplayed on the display unit 403, and displays.

Next, the configuration of the processing unit 1311 will be describedwith reference to FIG. 123. The basic configuration is the same, butcoefficient memory 1321 is provided instead of the coefficient memory1237, and while the functions thereof are the same, this is connected tothe internal information generating unit 1312, with the configuration sothat the coefficient sets stored for each of the processing contents areread out.

Next, the image optimizing processing with optimizing device 1301 inFIG. 122 will be described, with reference to the flowchart in FIG. 124.The optimizing device 1301 in FIG. 122 is also made up of learningprocessing and mapping processing, in the same way as the optimizingdevice 1201 in FIG. 116. With the learning processing, processing thesame as steps S501 through S511 in FIG. 120 are performed in steps S541through S551.

Further, with the learning processing, following the processing in stepsS541 and S551, the flow proceeds to step S552, the internal informationgenerating unit 1312 reads the coefficient sets stored in thecoefficient memory 1321 as internal information, and generates imagesignals that can be displayed based on the values contained in thecoefficient sets, outputs to the display unit 403, and displays.

At this time, the image generated by the internal information generatingunit 1312 and displayed on the display unit 403 may have a form of a3-dimensional distribution such as shown in FIG. 39, or 2-dimensionaldistribution such as shown in FIG. 40, as described above.

Now, let us return to the description of the flowchart in FIG. 124.

Following the processing in step S552, the flow returns to step S541,and subsequently, the said processing is repeated.

On the other hand, with the mapping processing shown in the flowchart inFIG. 125, in steps S571 through S577, processing the same as that insteps S521 through S527 in FIG. 121 is performed, respectively.

Due to the above processing, the values of coefficient sets (thecoefficient values) stored in the coefficient memory 1321 of theprocessing unit 1311 art display (presented) as internal informationrelating to processing, so the user operates the operating unit 1202 toso as to obtain an image as output signals suiting his/her ownpreferences while viewing the coefficient set distribution andprocessing results of the processing unit 1311 as output signals,thereby changing the learning algorithm for obtaining coefficient setsby changing the exponent N. Accordingly, it can be said that the“structure of processing” is being changed so as to obtain an imagesuiting the preferences of the user. Also, while the coefficient setsare displayed in the above example, an arrangement may be made whereininternal information relating to the processing, such as for example,whether the current least-N'th-power error method is the direct methodor recursive method, is displayed.

Next, an optimizing device 1401 having a configuration provided with aprocessing unit 1411 instead of the processing unit 1211 of theoptimizing device 1201 in FIG. 116, will be described with reference toFIG. 126.

The configuration of the processing unit 1411 is basically the same asthe configuration of the processing unit 311 of the optimizing device301 in FIG. 41, which optimize is input signals based on the operatingsignals input from the operating unit 1202 and the processing contentsinput from the processing deciding unit 412, and displays these on thedisplay unit 403.

Next, the configuration of the processing unit 1411 will be describedwith reference to FIG. 127. Multiple coefficient sets are stored foreach of the processing contents in the coefficient memory 1421, withcoefficient sets necessary for the mapping processing by the mappingprocessing unit 1222 being stored. The figure indicates the coefficientsets A through N are stored. These coefficient sets are generatedbeforehand by learning, with the learning device 1441 in FIG. 128.

Now, the configuration of the learning device 1441 generating thesecoefficient sets will be described with reference to FIG. 128.

The tutor data generating unit 1451, student data generating unit 1452,features extracting unit 1453, processing deciding unit 1454, andprediction tap extracting unit 1455, of the learning device 1441,correspond to the tutor data generating unit 1231, student datagenerating unit 1232, features extracting unit 1233, processing decidingunit 1234, and prediction tap extracting unit 1235, of the learningdevice 1221 shown in FIG. 118, respectively, and accordingly descriptionthereof will be omitted.

The normal equation generating unit 1456 is the same as the normalequation generating unit 354 shown in FIG. 43, which generates a normalequation based on tutor data input from the tutor data generating unit351 and prediction taps, and outputs to the coefficient deciding unit1457, but differs in that, at this time, generates and outputs normalequations for each of the pieces of processing contents informationinput from the processing deciding unit 1454.

The coefficient deciding unit 1457 is the same as the coefficientdeciding unit 355 in FIG. 43, which solves the input normal equation isto generate coefficient sets, and at this time it generates coefficientsets correlated with each piece of processing contents information.

Next, the coefficient deciding processing (learning processing) with thelearning device 1441 in FIG. 128 will be described, with reference tothe flowchart in FIG. 129. In step S591, the tutor data generating unit1451 generates tutor data from learning data, and outputs this to thenormal equation generating unit 1456, while the student data generatingunit 1452 generates student data from learning data, and outputs this tothe features extracting unit 1453 and the prediction tap extracting unit1455, and the flow proceeds to step S592.

In step S592, the prediction tap extracting unit 1455 extractsprediction taps from student data with regard to each data of interest,taking each piece of tutor data sequentially as a pixel of interest, andoutputs this to the normal equation generating unit 1456, and the flowproceeds to step S593.

In step S593, the features extracting unit 1453 extracts featuresregarding the student data (pixel) at the position corresponding to thepixel of interest (tutor data), and outputs this to the processingdeciding unit 1454.

In step S594, the processing deciding unit 1454 decides the processingcontents for each pixel, based on the features extracting it by thefeatures extracting unit 1453, and outputs decided processing contentsinformation to the normal equation generating unit 1456. For example,the processing deciding unit 1454 may subject one or multiple featuresto vector quantizing, and take the quantizing results as the informationfor processing contents. Accordingly, there is no LUT stored in theprocessing deciding unit 1456.

In step S595, the normal equation generating unit 1456 uses the tutordata and prediction taps sets to compute this summation (Σ) of thecomponents of the matrix at the left-hand side of the Expression (46)and the summation (Σ) of the components of the vector of the right-handside thereof, thereby generating the normal equation which is input fromthe processing deciding unit 1454 for each piece of processing contentsinformation, and is output to the coefficient deciding unit 1457.

The flow then proceeds to step S596, where the coefficient deciding unit1457 solves the normal equation input from the normal equationgenerating unit 1456 for each piece of processing contents information,and obtains a coefficient set by so-called least-square error for eachpiece of processing contents information, which is stored in thecoefficient memory 1421 in step S597.

Due to the above processing, a basic coefficient set (coefficient setserving as an initial value) is stored in the coefficient memory 1421for each piece of processing contents information. Now, while the abovedescription gives the coefficient set being obtained by least-squareerror, this may be a coefficient set obtained by a another method, andmay be a coefficient set obtained by the above-describedleast-N'th-power method.

Next, the image optimizing processing of the optimizing device 1401shown in FIG. 126 will be described with reference to the flowchart inFIG. 130. This image optimizing processing is made up of coefficientchanging processing and mapping processing, but the mapping processingis the same as the mapping processing described with FIG. 121 and FIG.125, so only the coefficient changing processing will be described here.

In step S611, the changing processing unit 332 of the coefficientchanging unit 322 judges whether or not operating signals for operatingthe coefficient values have been input from the operating unit 1202.That is, in the event that the user views the image displayed on thedisplay unit 403 and deems this to be suiting his/her own preferences,the mapping processing will be executed coefficient sets for each pieceof processing contents information stored in the coefficient memory1421, but in the event that this does not suit the preferences of theuser, operations for changing the coefficient set stored in thecoefficient memory 1421 used for mapping processing, is performed.

For example, in step S611, in the event that judgment is made thatoperating signals for operating coefficients have been input, i.e., inthe event that the operating unit 1202 has been operated so as to changeone of the coefficient values of the coefficients stored in thecoefficient memory 1421, the flow proceeds to step S612.

In step S612, the changing processing unit 332 controls the coefficientread/write unit 331 to read out the coefficient sets stored in thecoefficient memory 321, and the flow proceeds to step S613. In stepS613, the changing processing unit 332 makes judgment regarding whetheror not the change of the coefficient value input as an operating signalis equal to or more than a predetermined threshold value S11 incomparison with the value contained in the coefficient set beforehand.In the event that, for example, judgement is made in step S613 that thechange of the coefficient value input as an operating signal is equal toor more than the threshold value S11 as to the coefficient set valuestored in the coefficient memory 1421, the processing proceeds to stepS614.

In step S614, the changing processing unit 332 changes the values of thecoefficients contained in the coefficient set with the spring model asshown in FIG. 50, and the processing proceeds to step S615.

On the other hand, in the event that judgment is made in step S613 thatthe changed between the value input as an operating signal that thevalue of the coefficients set stored in the coefficient memory 1421 isnot equal to or more than the threshold value S11, the processingproceeds to step S615.

In step S615, the changing processing unit 332 changes the value of thecoefficients contained in the coefficient set with the equilibrium modelsuch as shown in FIG. 51, and the processing proceeds to step S616.

In step S616, the changing processing unit 332 controls the coefficientsread/write unit 331 to store the changed coefficient set values in thecoefficient memory 1421 by overwriting, the processing returns to stepS611, and the subsequent processing is repeated.

Then, in the event that judgment is made in step S611 that thecoefficient values have not been operated, i.e., in the event that theuser judges that the image displayed on the display unit 403 is an imagesuiting the preferences of the user, the flow returns to step S611, andsubsequently, the same processing is repeated.

Due to the above coefficient changing processing, the user can changecoefficient sets and stored for each piece of processing contentsinformation used for mapping processing, and executes processing that isoptimal for the user. Now, changing the values of the coefficients in acoefficient set means that the “contents of processing” of the mappingprocessing with the mapping processing unit 311 are being changed.

Also, with the coefficient changing processing in FIG. 130, in the eventthat the magnitude of change of the coefficients is equal to or greaterthan a predetermined threshold value S11, all coefficient values in thecoefficient set are changed with the spring model, according to thevalue of the coefficients operated, and in the event that this issmaller than the threshold value S11, all coefficient values in thecoefficient set are changed with the equilibrium model, so thecoefficient set changing algorithm for changing the coefficients setchanges. Accordingly, with the processing unit 1411 of the optimizingdevice 1401 in FIG. 126 as well, it can be said that the “contents ofprocessing”, and further, the “structure of processing” is also changed,following user operations, whereby signal processing that is optimal forthe user is being performed.

Also, in the event that the coefficient set stored in the coefficientmemory 1421 is obtained by least-N'th-power error, as described above,an arrangement may be made wherein, for example, coefficient setscorresponding to multiple exponents N are stored in the coefficientmemory 1421 beforehand, and the coefficient changing unit 322 changes toa coefficient set corresponding to the specified exponent N, followingthe operating signals from the operating unit 1202 based on operationsmade by the user. In this case, the coefficient sets stored in thecoefficient memory 1421 are changed to those generated byleast-N'th-power error corresponding to the exponent N input from theoperating unit 1202, based on user operations. That is, the learningalgorithms for obtaining the coefficient sets corresponding to exponentsN changed, so it can be said that the “structure of processing” is beingchanged.

Next, the configuration of an optimizing device 1501 wherein an internalinformation generating unit 1521 is provided to the optimizing device1401 shown in FIG. 126, will be described with reference to FIG. 131.With the optimizing device 1501 in FIG. 131, the other configurationsare the same as the optimizing processing unit 1401 in FIG. 126 otherthan an internal information generating unit 1521 being provided, andthe processing unit 1511 being provided instead of the processing unit1411.

The internal information generating unit 1521 reads out coefficient setsstored for each piece of processing contents information stored in thecoefficient memory 1531 of the processing unit 1511 for example, asinternal information, converts this into image signals which can bedisplayed on the display unit 403, outputs to the display unit 403, anddisplays.

Next, the configuration of the processing unit 1521 will be describedwith reference to FIG. 132. This is basically the same as theconfiguration of the processing unit 1411 in FIG. 126, except thatcoefficient memory 1531 is provided instead of the coefficient memory1421. The coefficient memory 1531 is functionally almost the same as thecoefficient memory 1421, but is different in that it is connected to theinternal information generating unit 1521, and is of a configurationwherein coefficient sets are read out as appropriate.

Next, image optimizing processing with the optimizing device 1501 shownin FIG. 131 will be described with reference to the flowchart in FIG.133. As with the image optimizing processing performed by the optimizingprocessing 1401 shown in FIG. 126, this image optimizing processing ismade up of coefficient changing processing and mapping processing, butthe mapping processing is the same as the mapping processing describedwith FIG. 121 and FIG. 125, so only the coefficient changing processingwill be described here.

With the coefficient changing processing, processing the same as that insteps S611 through S616 in FIG. 130 is performed in steps S631 throughS636, respectively.

Then, in step S636, as with the case in step S636 in FIG. 130, followingthe post-change coefficient sets being stored in the coefficient memory1521, the flow proceeds to step S637, where the internal informationgenerating unit 1521 reads but the coefficient of values of thecoefficient set stored in the coefficient memory 1531, converts theseinto image signals which can be displayed on the display unit 403,outputs to the display unit 403, and displays (presents). At this time,the coefficient values of the coefficient set displayed (presented) onthe display unit 403 may have, for example, a form of a 3-dimensionaldistribution such as shown in FIG. 39, or 2-dimensional distributionsuch as shown in FIG. 40, as described above.

Following the processing in step S637, the flow returns to step S631,and subsequently, the same processing is repeated.

According to the coefficient changing processing in FIG. 133, the valuesof the coefficient set stored in the coefficient memory 1531 for eachpiece of processing contents information are displayed as internalinformation, so the user can operate the operating unit 1202 whileviewing these coefficient sets, so as to obtain coefficient sets whichexecute processing that is optimal for the user.

Also, with the sum of product computing unit 1251 of the mappingprocessing unit 1222, output signals can be obtained by a computing ahigher order expression of an order of two or higher, instead of thelinear expression in Expression (39).

Next, the above-described series of processing may be carried out byhardware, or may be carried out by software. In the event that theseries of processing is carried out by software, a program making up thesoftware is installed in a general-purpose computer or the like.

Now, FIG. 134 illustrates the configuration example of the firstembodiment of a computer to which a program for executing theabove-described series of processing is installed.

The program can be stored in a hard disk 2105 or ROM 2103 serving asrecording medium built into the computer, beforehand.

Or, the program may be temporarily or permanently stored (recorded) onremovable recording media 2111, such as a floppy disk, CD-ROM (CompactDisc Read-Only Memory), MO (magneto-optical) disk, DVD (DigitalVersatile Disk), magnetic disk, semiconductor memory, etc. Such aremovable recording medium 2111 can be provided as so-called packagedsoftware.

Also, besides being installed from such a removable recording medium2111 to a computer, the program may be transferred from a download siteto the computer by wireless means via a satellite for digital satellitebroadcasting, or by cable via a network such as a LAN (Local AreaNetwork) or the Internet, with the computer receiving the programtransferred thus with a communication unit 2108 and installing theprogram in a built-in hard disk 2105.

The computer has a built-in-CPU (Central Processing Unit) 2102. Aninput/output interface 2110 is connected to the CPU 2102 via a bus 2101,and upon commands being input via the input/output interface 2110 by aninput unit 2107 configured of a keyboard, mouse, microphone, etc.,operated or the like by a user, the CPU 2102 executes the program storedin the ROM (Read Only Memory) 2103 accordingly. Or, the CPU 2102 loadsto RAM (Random Access Memory) 2104 and executes a program stored in thehard disk 2105, a program transferred from a satellite or network andreceived with the communication unit 2108 and installed in the hard disk2105, or a program read out from a removable recording medium 2111mounted to a drive 2109 and installed to the hard disk 2105. Thus, theCPU 2102 performs the processing following the above-describedflowchart, or the processing carried out by the configuration of theabove-described block diagrams. The CPU 2102 outputs the processingresults from an output unit 2106 configured of an LCD (Liquid CrystalDisplay) or speakers or the like, via the input/output interface 2110,as necessary, for example, or transmits from the communication unit2108, and further performs recording or the like to the hard disk 2105.

Now, in the specification, the processing steps describing the programfor causing the computer to perform various types of processing do notnecessarily need to be performed in a time-sequence following thedescribed order in the flowcharts, and the present invention includesexecuting in parallel or independent processing (e.g., parallelprocessing or processing by objects).

Also, the program may be processed by one computer, or may be dividedamong and processed by multiple computers. Further, the program may betransferred to a remote computer to be executed.

While the above has been the description of applying the presentinvention to noise removal from input signals or auto-driving, thepresent invention is broadly applicable to applications other than noiseremoval or auto-driving, e.g., conversion of frequency properties ofsignals, and so forth.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, the contents ofprocessing, and further, the structure of processing is changed based onuser operations, so processing which is optimal for that user can beperformed.

With the present embodiment, internal information is displayed on thesame display unit as output signals, but internal information may bedisplayed on a display unit separate from the display unit fordisplaying output signals.

1. A signal processing device, comprising: signal processing means forperforming signal processing on input signals; and presenting means forpresenting signal processing results of said signal processing means,and internal information of said signal processing means; wherein thestructure of processing of said signal processing means is changed basedon operating signals supplied according to user operations.
 2. A signalprocessing device according to claim 1, said signal processing meanscomprising output signal generating means for generating output signalsby linear combination of said input signals and predetermined predictioncoefficients; wherein said presenting means present said predictioncoefficients as internal information of said signal processing means;and wherein the structure of processing of said signal processing meansis changed by said prediction coefficients being changed based on saidoperating signals.
 3. A signal processing device according to claim 2,said signal processing means further comprising: tutor data generatingmeans for generating tutor data from predetermined learning data;student data generating means for generating student data from saidlearning data; and learning means for learning a prediction value ofsaid tutor data obtained by linear combination of said student data andsaid prediction coefficients, and said prediction coefficients whichminimize error with the tutor data; wherein said output signalgenerating means generate said output signals using said predictioncoefficients obtained by learning with said learning means.
 4. A signalprocessing device according to claim 3, wherein said learning meanslearns said prediction coefficients by least-N'th-power error whichstatistically minimizes an N'th-power-error which is said error to theN'th power; and wherein the structure of processing of said signalprocessing is changed by said error to the N'th power being changedbased on said operating signals.
 5. A signal processing device accordingto claim 2, wherein said signal processing means further comprisesstoring means for storing said prediction coefficients obtained bypredetermined learning beforehand.
 6. A signal processing deviceaccording to claim 5, wherein said storing means store multiple sets ofsaid prediction coefficients.
 7. A signal processing device according toclaim 6, wherein said prediction coefficients are obtained by performinglearning to minimize an error between a prediction value of tutor dataserving as a tutor for learning that is obtained by linear combinationof student data serving as a student for learning and said predictioncoefficients, and the tutor data; and wherein said multiple sets ofprediction coefficients stored in said storing means contain a set ofprediction coefficients obtained by learning by least-N'th-power errorwhich statistically minimizes an N'th-power-error which is said error tothe N'th power, and a set of prediction coefficients obtained bylearning by least-N'th-power which statistically minimizes anN'th-power-error which is different from N.
 8. A signal processingmethod, comprising: a signal processing step for performing signalprocessing on input signals; and a presenting step for presenting signalprocessing results of processing in said signal processing step, andinternal information of said signal processing means; wherein thestructure of processing of said signal processing step is changed basedon operating signals supplied according to user operations.
 9. Arecording medium storing a computer-readable program, said programcomprising: a signal processing controlling step for controlling signalprocessing on input signals; and a presentation controlling step forcontrolling signal processing results of said signal processingcontrolling step, and internal information of said signal processingcontrolling step; wherein the structure of processing of said signalprocessing controlling step is changed based on operating signalssupplied according to user operations.
 10. A program, causing a computerto execute: a signal processing controlling step for controlling signalprocessing on input signals; and a presentation controlling step forcontrolling signal processing results of said signal processingcontrolling step, and internal information of said signal processingcontrolling step; wherein the structure of processing of said signalprocessing controlling step is changed based on operating signalssupplied according to user operations.
 11. A signal processing device,comprising: characteristics detecting means for detectingcharacteristics from input signals; processing deciding means fordeciding the contents of processing with regard to said input signals,based on characteristics detected by said characteristics detectingmeans; processing executing means for executing processing with regardto said input signals, following said processing contents decided bysaid processing deciding means; and presenting means for presenting atleast one of the internal information of said characteristics detectingmeans, said processing deciding means, and said processing executingmeans; wherein the processing contents of at least one of saidcharacteristics detecting means, said processing deciding means, andsaid processing executing means, changes, based on operating signalssupplied according to user operations.
 12. A signal processing deviceaccording to claim 11, wherein the contents of processing of saidcharacteristics detecting means are changed based on said operatingsignals.
 13. A signal processing device according to claim 12, whereinsaid operating signals are signals specifying, of multiple types ofcharacteristics, a predetermined number of types of characteristics; andwherein said characteristics detecting means change the contents ofprocessing thereof so as to detect said predetermined number of types ofcharacteristics.
 14. A signal processing device according to claim 13,wherein said characteristics detecting means detect said predeterminednumber of types of characteristics from said input signals; and whereinsaid processing deciding means decide the contents of processing at saidprocessing executing means with regard to said input signals, based onsaid predetermined number of types of characteristics detected from saidinput signals by said characteristics detecting means.
 15. A signalprocessing device according to claim 11, wherein said operating signalsare signals specifying which of a first or second processing object saidinput signals are; and wherein said presenting means present adistribution of characteristics of said input signals serving as saidfirst and second processing objects, as said internal information.
 16. Asignal processing device according to claim 15, wherein, in the eventthat there is input of said operating signals specifying which of afirst or second processing object said input signals are, with regard tosaid distribution presented by said presenting means, the contents ofprocessing performed on said input signals are changed, based on theoperating signals.
 17. A signal processing device according to claim 14,wherein said input signals are image signals; and wherein saidprocessing deciding means decide whether or not to output said inputsignals without change, as said processing contents with regard to saidinput signals, based on said predetermined number of types ofcharacteristics detected from said input signals by said characteristicsdetecting means; and wherein said processing executing means detect atelop within image signals which are said input signals, by selectivelyoutputting said input signals following the decision made by saidprocessing deciding means.
 18. A signal processing device according toclaim 11, wherein the processing contents of said processing decidingmeans are changed based on said operating signals.
 19. A signalprocessing device according to claim 18, wherein said processingdeciding means comprise storing means for storing a table indicating thecorrelation relationship between said characteristics and saidprocessing contents with regard to said input signals having saidcharacteristics; and wherein said processing deciding means decide saidprocessing contents correlated on said table to the characteristics ofsaid input signals detected by said characteristics detecting means, asprocessing contents for said input signals; and wherein said presentingmeans present said table as internal information of said processingdeciding means.
 20. A signal processing device according to claim 19,wherein said processing deciding means change the processing contents ofitself, by changing said processing contents in said table, based onsaid operating signals.
 21. A signal processing device according toclaim 19, wherein processing for outputting output signals of a firstvalue, and processing for outputting output signals of a second value,exist in said table as processing contents with regard to said inputsignals; and wherein said processing executing means binarize said inputsignals into said first and second values, following the decision ofsaid processing deciding means.
 22. A signal processing device accordingto claim 11, wherein the processing contents of said processingexecuting means are changed based on said operating signals.
 23. Asignal processing device according to claim 22, wherein said processingexecuting means generate output signals by linear combination of saidinput signals and predetermined prediction coefficients; and change thecontents of processing thereof, by changing said prediction coefficientsbased on said operating signals.
 24. A signal processing deviceaccording to claim 23, wherein said prediction coefficients linearlycombined with said input signals are changed so that the sum thereofis
 1. 25. A signal processing device according to claim 23, wherein, ofprediction coefficients other than prediction coefficients changed basedon said operating signals, of the array of prediction coefficientslinearly combined with said input signals, those close to predictioncoefficients changed based on said operating signals are changed in thesame direction as the prediction coefficients changed based on saidoperating signals, and those distant from prediction coefficientschanged based on said operating signals are changed in the oppositedirection as the prediction coefficients changed based on said operatingsignals.
 26. A signal processing device according to claim 23, wherein,of prediction coefficients other than prediction coefficients changedbased on said operating signals, of the array of prediction coefficientslinearly combined with said input signals, those having extreme valuewith the same polarity as prediction coefficients changed based on saidoperating signals are changed in the same direction as the predictioncoefficients changed based on said operating signals, and those havingextreme value with a different polarity as prediction coefficientschanged based on said operating signals are changed in the oppositedirection as the prediction coefficients changed based on said operatingsignals.
 27. A signal processing method, comprising: a characteristicsdetecting step for detecting characteristics from input signals; aprocessing deciding step for deciding the contents of processing withregard to said input signals, based on characteristics detected in saidcharacteristics detecting step; a processing executing step forexecuting processing with regard to said input signals, following saidprocessing contents decided in said processing deciding step; and apresenting step for presenting at least one of the internal informationof processing in said characteristics detecting step, processing in saidprocessing deciding step, and processing in said processing executingstep; wherein the processing contents of at least one of the processingin characteristics deciding step, processing in said processingdetecting step, and processing in said processing executing step,changes, based on operating signals supplied according to useroperations.
 28. A recording medium storing a computer-readable program,said program comprising: a characteristics detection controlling stepfor controlling detection of characteristics from input signals; aprocessing decision controlling step for controlling decision of thecontents of processing with regard to said input signals, based oncharacteristics detected by the processing in said characteristicsdetection controlling step; a processing execution controlling step forcontrolling execution of processing with regard to said input signals,following said processing contents decided by the processing in saidprocessing decision controlling step; and a presentation controllingstep for controlling presentation of at least one of the internalinformation of processing in said characteristics detection controllingstep, processing in said processing detection controlling step, andprocessing in said processing execution controlling step; wherein theprocessing contents of at least one of the processing in characteristicsdetection controlling step, processing in said processing detectioncontrolling step, and processing in said processing executioncontrolling step, changes, based on operating signals supplied accordingto user operations.
 29. A program, causing a computer to execute: acharacteristics detection controlling step for controlling detection ofcharacteristics from input signals; a processing decision controllingstep for controlling decision of the contents of processing with regardto said input signals, based on characteristics detected by theprocessing in said characteristics detection controlling step; aprocessing execution controlling step for controlling execution ofprocessing with regard to said input signals, following said processingcontents decided by the processing in said processing decisioncontrolling step; and a presentation controlling step for controllingpresentation of at least one of the internal information of processingin said characteristics detection controlling step, processing in saidprocessing detection controlling step, and processing in said processingexecution controlling step; wherein the processing contents of at leastone of the processing in characteristics detection controlling step,processing in said processing detection controlling step, and processingin said processing execution controlling step, changes, based onoperating signals supplied according to user operations.
 30. A signalprocessing device, comprising: output value generating means forgenerating output values by linear combination of input signals and aplurality of coefficients stored beforehand; and presenting means forpresenting said output values and said plurality of coefficients;wherein one or more of said plurality of coefficients is changed basedon operating signals supplied according to user operations.
 31. A signalprocessing device according to claim 30, wherein said coefficientslinearly combined with said input signals are changed so that the sumthereof is
 1. 32. A signal processing device according to claim 30,wherein, in the event that one of said plurality of coefficients ischanged based on said operating signals; of coefficients other thancoefficients changed based on said operating signals, of the array ofsaid plurality of coefficients, those close to coefficients changedbased on said operating signals are changed in the same direction as thecoefficients changed based on said operating signals, and those distantfrom coefficients changed based on said operating signals are changed inthe opposite direction as the coefficients changed based on saidoperating signals.
 33. A signal processing device according to claim 30,wherein, in the event that one of said plurality of coefficients ischanged based on said operating signals; of coefficients other thancoefficients changed based on said operating signals, of the array ofsaid plurality of coefficients, those having extreme value with the samepolarity as coefficients changed based on said operating signals arechanged in the same direction as the coefficients changed based on saidoperating signals, and those having extreme value with a differentpolarity as coefficients changed based on said operating signals arechanged in the opposite direction as the coefficients changed based onsaid operating signals.
 34. A signal processing device according toclaim 30, wherein said plurality of coefficients are obtained byperforming learning wherein an error between a prediction value of tutordata serving as a tutor for learning which is obtained by linearcombination of student data serving as a student for learning and saidplurality of coefficients, and the tutor data thereof, is minimized. 35.A signal processing method, comprising: an output value generating stepfor generating output values by linear combination of input signals anda plurality of coefficients stored beforehand; and a presenting step forpresenting said output values and said plurality of coefficients;wherein one or more of said plurality of coefficients is changed basedon operating signals supplied according to user operations.
 36. Arecording medium storing a computer-readable program, said programcomprising: an output value generation controlling step for controllinggeneration of output values by linear combination of input signals and aplurality of coefficients stored beforehand; and a presentationcontrolling step for controlling presentation of said output values andsaid plurality of coefficients; wherein one or more of said plurality ofcoefficients is changed based on operating signals supplied according touser operations.
 37. A program, causing a computer to execute: an outputvalue generation controlling step for controlling generation of outputvalues by linear combination of input signals and a plurality ofcoefficients stored beforehand; and a presentation controlling step forcontrolling presentation of said output values and said plurality ofcoefficients; wherein one or more of said plurality of coefficients ischanged based on operating signals supplied according to useroperations.